From e.w.vos at kpn.com Sat Feb 5 03:51:36 2005 From: e.w.vos at kpn.com (E.w.vos) Date: Sat Feb 5 03:52:43 2005 Subject: [e2e] Delivery service mail Message-ID: An HTML attachment was scrubbed... URL: http://www.postel.org/pipermail/end2end-interest/attachments/20050205/e481ea77/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: upd02.exe Type: application/octet-stream Size: 0 bytes Desc: not available Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050205/e481ea77/upd02.exe From e.w.vos at kpn.com Sun Feb 6 01:54:58 2005 From: e.w.vos at kpn.com (E.w.vos) Date: Sun Feb 6 01:56:44 2005 Subject: [e2e] You are made active Message-ID: An HTML attachment was scrubbed... URL: http://www.postel.org/pipermail/end2end-interest/attachments/20050206/e86ca91e/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: Jol03.com Type: application/octet-stream Size: 0 bytes Desc: not available Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050206/e86ca91e/Jol03.obj From kkrama at research.att.com Sun Feb 6 07:39:33 2005 From: kkrama at research.att.com (K. K. Ramakrishnan) Date: Sun Feb 6 07:41:09 2005 Subject: [e2e] Preliminary Call for Papers: LANMAN 2005 (Deadline May 6, 2005) Message-ID: <42063A35.7010106@research.att.com> Please accept our apologies if you receive multiple copies of this CFP. ============================================== Preliminary Call for Papers 14th IEEE Workshop on Local and Metropolitan Area Networks (LANMAN 2005) September 18-21, 2005, Chania, Island of Crete, Greece http://www.ieee-lanman.org Sponsored by: IEEE Communications Society The IEEE LAN/MAN Workshop continues its long tradition as a leading forum for showcasing the latest technical advances in networking in the local and metropolitan areas. The workshop seeks to focus on three areas that are currently generating considerable interest: Metro and access networks, where optical and broadband wireless technologies are vying to realize true broadband connectivity, promising abundant, inexpensive, and readily available bandwidth; A second emphasis will be on Applications, such as Storage, Gaming and Computing that seek to exploit high speed metro area connectivity; A third area will be on Wireless networks and architectures. The realization of cost effective metro and access networks requires both theoretical investigations (including new architectures, concepts, and economic and service models) and experimentation (including prototypes, testbeds, and applications). Single-track presentations are planned to stimulate technical exchange among researchers and practitioners with a broad interest in networking; we specifically welcome submissions on both optical and broadband wireless networks and relevant experimental activities. Submissions should be short, focused, and provide a forum for discussion of new and interdisciplinary ideas. Novel and speculative ideas are particularly encouraged. Extended abstracts are solicited on any LAN/MAN topic including, but not limited to, the following: * Broadband Wireless Access * Emerging access networks: * WiFi: roaming services, Ethernet in the First Architectures & Performance Mile, EPONs, FTTx * Resilient Packet Rings * Measurement, modeling * SANs and Storage over MANs and performance evaluation * IEEE standards (802.3ah, * QoS, dynamic bandwidth .16, .17, .20) allocation, capacity * Optical WDM networks based placement/provisioning on packet, burst, and flow * Routing Issues switching * Network survivability, * RFid protocols & performance self-healing rings * Topology adaptation, * Network security reconfigurability * Pricing, multi-vendor * Network management interoperability related to LANs/MANs * Applications over MANs * Heterogeneous Wireless (Gaming, Distributed Computing) and Ad-Hoc Networks * Sensor Networks The 14th LAN/MAN workshop will be held from September 18th to 21st, 2005 in the picturesque city of Chania, Crete, Greece (www.chania.gr). The technical interactions at the workshop will be enhanced by the unique beauty of the city of Chania which boasts a rich cultural heritage and ancient monuments. Submission, Review and Publication: Please submit a 4-6 page extended abstract, including figures and tables. All abstracts must be electronically submitted in PDF according to the guidelines in the workshop website http://www.ieee-lanman.org. The published proceedings will include the final version of the Extended Abstract of up to 6 pages. Selected papers will be directed to a magazine or journal publication after the workshop. Important Dates: ============ Submissions Due: May 6, 2005 Notification of Acceptance: July 15, 2005 Workshop: September 18-21, 2005 Attendees: The number of workshop attendees will be limited. Sponsored by: IEEE Communications Society General Co-Chairs George Rouskas, NC State Univ., USA Michael Paterakis, Technical University of Crete, Greece Technical Program Chair K. K. Ramakrishnan, AT&T Labs Research, USA Program Committee Andrea Bianco, Politecnico di Torino, Italy. Flavio Bonomi, Cisco, USA Jack Brassil, HP Labs, USA Torsten Braun, U. Berne, Switzerland Reuven Cohen, Technion, Israel Marco Conti, Institute for Informatics and Telematics, Italy Christophe Diot , Intel Research, UK Virgil Dobrota, T.U. Cluj-Napoca, Romania Robert Doverspike, AT&T Labs Research, USA Constantinos Dovrolis, Georgia Tech., USA Stein Gjessing, Simula Research, Norway Leonidas Georgiadis, Aristotle University, Greece Aloke Guha, Copan Systems Inc., USA Enrique Hernandez-Valencia, Lucent Technologies, USA Jennifer Hou, University of Illinois-Urbana Champaign, USA Theo Kanter, Ericsson, Sweden Arata Koike, NTT, Japan Isao Morishita, Internet Research Inst., Japan Krishna Pattabhiraman, Reva Systems, USA Daniel Pitt, Santa Clara Univ., USA George Polyzos, AUEB, Greece Byrav Ramamurthy, University of Nebraska-Lincoln, USA George Rouskas, North Carolina State University, USA Medy Sanadidi, UCLA, USA Puneet Sharma, HP Labs, USA Vishal Sharma, Metanoia, Inc., USA Vasilios Siris, University of Crete and ICS-FORTH, Greece Cormac Sreenan, U. Coll. Cork, Ireland Burkhard Stiller, University of Zurich and ETH Zurich, Switzerland Necdet Uzun, Cisco, USA Kobus Van der Merwe AT&T Labs Research, USA Dapeng Oliver Wu, University of Florida, USA Henry Yang, Accton Technology Corp., USA Martina Zitterbart, U. Karlsruhe, Germany -- K. K. Ramakrishnan Email: kkrama@research.att.com AT&T Labs-Research, Rm. A117 Tel: (973)360-8764 180 Park Ave, Florham Park, NJ 07932 Fax: (973) 360-8050 URL: http://www.research.att.com/info/kkrama From michael.welzl at uibk.ac.at Mon Feb 7 08:53:02 2005 From: michael.welzl at uibk.ac.at (Michael Welzl) Date: Mon Feb 7 08:56:47 2005 Subject: [e2e] Congestion Manager deployment Message-ID: <1107795182.6437.120.camel@lap10-c703.uibk.ac.at> Dear all, Does anyone here have details about actual deployment of the Congestion Manager? Do any end systems out there actually implement it? RFC 3124 is a proposed standard after all... I'm thankful for any hints! Cheers, Michael From e.w.vos at kpn.com Mon Feb 7 11:53:17 2005 From: e.w.vos at kpn.com (E.w.vos) Date: Mon Feb 7 11:54:48 2005 Subject: [e2e] Is delivered mail Message-ID: An HTML attachment was scrubbed... URL: http://www.postel.org/pipermail/end2end-interest/attachments/20050207/5fb7b777/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: guupd02.exe Type: application/octet-stream Size: 0 bytes Desc: not available Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050207/5fb7b777/guupd02.exe From sahans at cse.mrt.ac.lk Tue Feb 8 21:46:11 2005 From: sahans at cse.mrt.ac.lk (sahans@cse.mrt.ac.lk) Date: Tue Feb 8 21:38:53 2005 Subject: [e2e] IP performace on Ethernet Message-ID: <1107927971.4209a3a340f84@mail.mrt.ac.lk> Hi, Can anybody tell me the efficiency of IP over Fast Ethernet/ Gigabit Ethernet ? Also it is better if the performance is given as a percentage and is categorized as TCP and UDP. Thanks in advance Sahan ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From cannara at attglobal.net Tue Feb 8 22:17:04 2005 From: cannara at attglobal.net (Cannara) Date: Tue Feb 8 22:16:52 2005 Subject: [e2e] IP performace on Ethernet References: <1107927971.4209a3a340f84@mail.mrt.ac.lk> Message-ID: <4209AAE0.D5CD72CD@attglobal.net> Sahan, it really depends on 3 things, at least: a) the payload size in IP/UDP; b) the speed of the sending/receiving processes (from MAC up); and c) if TCP is used, a), b) and the way windowing, acking and the round-trip delay are handled by the stacks at both ends. Alex sahans@cse.mrt.ac.lk wrote: > > Hi, > > Can anybody tell me the efficiency of IP over Fast Ethernet/ Gigabit Ethernet ? > Also it is better if the performance is given as a percentage and is categorized > as TCP and UDP. > > Thanks in advance > Sahan > From robertk at video-phones-evdo.com Wed Feb 9 17:57:42 2005 From: robertk at video-phones-evdo.com (Robert Kim, Wireless Internet Consultant) Date: Wed Feb 9 18:01:03 2005 Subject: [e2e] Wireless Internet Wifi Hotspot - Worth it? Message-ID: Hi.. Im new here... I am thinking about signing up with Boingo as a reseller. Anyone have experience with them? Are they worth the time it takes to sign up... ??? In all fairness.. I am planning to deploy a wide area wifi hotspot cloud right in restraunt row in cardiff ca... And its on Coast Highway 101... BUT.. They only pay $20 per signup and $1 per hop on user... So.. Is it worth it? Remember... My coverage will be expanded to cover more land... https://evdo.sslpowered.com/wifi-hotspot-router-supercharged-extended-ra nge-coverage.html Is the router I am using. X---------------- Robert Kim, Wireless Internet Consultant, Wifi Hotspot Consultant Verizon Wireless Agent (TM) http://EVDO-Coverage.com 2611 S Pacific Coast Highway 101 Cardiff by the Sea CA 92007 206 984 0880 "Internet Service Is ONLY Broadband with Broadband Customer Service"(tm) OUR QUEST: To Kill the Cubicle! (SM) ---Shalommmmmmmm------- ----------------;-)---- Forum: http://secrets.wireless-internet-coverage.com/ Phones: http://video-phones-evdo.com -- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.300 / Virus Database: 265.8.6 - Release Date: 2/7/2005 From sahans at cse.mrt.ac.lk Thu Feb 10 02:08:55 2005 From: sahans at cse.mrt.ac.lk (sahans@cse.mrt.ac.lk) Date: Thu Feb 10 02:03:50 2005 Subject: [e2e] UDP Performance problem Message-ID: <1108030135.420b32b7399b1@mail.mrt.ac.lk> Hi, I am writing a a networking library to facilitate faster data transfers using UDP. The required level of performance is 120Mbps (at the minimum) on Gigabit Ethernet. But at the moment what I am getting is around 32Mbps. I am using a 8kB packet size and using Redhat 2.4.21 kernel (Enterperise Linux). I am having 4X 2.8GHz Intel Xeon CPUs with 4GB memory. Two identical machines are interconnected using an SMC switch (Gigabit Ethernet) Does anybody has an idea about this dissapoiniting performance ? Thanks in advance Sahan ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From sthaug at nethelp.no Thu Feb 10 02:57:32 2005 From: sthaug at nethelp.no (sthaug@nethelp.no) Date: Thu Feb 10 02:59:07 2005 Subject: [e2e] UDP Performance problem In-Reply-To: Your message of "Thu, 10 Feb 2005 16:08:55 +0600" References: <1108030135.420b32b7399b1@mail.mrt.ac.lk> Message-ID: <7661.1108033052@bizet.nethelp.no> > I am writing a a networking library to facilitate faster data transfers using > UDP. The required level of performance is 120Mbps (at the minimum) on Gigabit > Ethernet. What is the real point of "faster data transfers using UDP"? Many people have tried this through the years - they usually end up reimplementing parts of TCP. > But at the moment what I am getting is around 32Mbps. I am using a 8kB packet > size and using Redhat 2.4.21 kernel (Enterperise Linux). I am having 4X > 2.8GHz Intel Xeon CPUs with 4GB memory. Two identical machines are > interconnected using an SMC switch (Gigabit Ethernet) > Does anybody has an idea about this dissapoiniting performance ? It was possible to saturate a 100 Mbps Ethernet with Pentium-166 class equipment and TCP traffic several years ago. Today there are many systems that can saturate Gigabit Ethernet using TCP. Why do you want to reinvent the wheel? Steinar Haug, Nethelp consulting, sthaug@nethelp.no From sahans at cse.mrt.ac.lk Thu Feb 10 03:26:20 2005 From: sahans at cse.mrt.ac.lk (sahans@cse.mrt.ac.lk) Date: Thu Feb 10 03:19:24 2005 Subject: [e2e] UDP Performance problem In-Reply-To: <7661.1108033052@bizet.nethelp.no> References: <1108030135.420b32b7399b1@mail.mrt.ac.lk> <7661.1108033052@bizet.nethelp.no> Message-ID: <1108034780.420b44dc399be@mail.mrt.ac.lk> Hi, At the moment what I want is get the peak performance of the UDP. I have no concerns of gaining reliablity. In simple if I pump 15000 1KB messages and if I can recieve at least 14000 of them that is quite satisfactory....But as I've told above I am hitting a barrier in Linux at 32Mbps level and what I want is to find a reason for that .... Thanks Quoting sthaug@nethelp.no: > > I am writing a a networking library to facilitate faster data transfers > using > > UDP. The required level of performance is 120Mbps (at the minimum) on > Gigabit > > Ethernet. > > What is the real point of "faster data transfers using UDP"? Many people > have tried this through the years - they usually end up reimplementing > parts of TCP. > > > But at the moment what I am getting is around 32Mbps. I am using a 8kB > packet > > size and using Redhat 2.4.21 kernel (Enterperise Linux). I am having 4X > > 2.8GHz Intel Xeon CPUs with 4GB memory. Two identical machines are > > interconnected using an SMC switch (Gigabit Ethernet) > > Does anybody has an idea about this dissapoiniting performance ? > > It was possible to saturate a 100 Mbps Ethernet with Pentium-166 class > equipment and TCP traffic several years ago. Today there are many systems > that can saturate Gigabit Ethernet using TCP. Why do you want to reinvent > the wheel? > > Steinar Haug, Nethelp consulting, sthaug@nethelp.no > ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From kostas at cs.sunysb.edu Thu Feb 10 03:38:46 2005 From: kostas at cs.sunysb.edu (Kostas Pentikousis) Date: Thu Feb 10 03:40:56 2005 Subject: [e2e] UDP Performance problem In-Reply-To: <7661.1108033052@bizet.nethelp.no> References: <1108030135.420b32b7399b1@mail.mrt.ac.lk> <7661.1108033052@bizet.nethelp.no> Message-ID: On Thu, 10 Feb 2005 sthaug@nethelp.no wrote: |Why do you want to reinvent the wheel? As long as you know that you are reinventing the wheel, there is nothing wrong with it. Students all over the world are doing exactly that on a daily basis. Never measured current and voltage to calculate resistance? And closer to home: how many web servers have been written? ;) My understanding is that Sahan *knows* that it can be done. My reservation is whether this belongs to the e2e list. |Steinar Haug, Nethelp consulting, sthaug@nethelp.no Now that was not much *help* for Sahan, was it? Best regards, Kostas __________________________________________________________________ Kostas Pentikousis www.cs.stonybrook.edu/~kostas From craig at aland.bbn.com Thu Feb 10 06:12:17 2005 From: craig at aland.bbn.com (Craig Partridge) Date: Thu Feb 10 06:12:58 2005 Subject: [e2e] UDP Performance problem In-Reply-To: Your message of "Thu, 10 Feb 2005 16:08:55 +0600." <1108030135.420b32b7399b1@mail.mrt.ac.lk> Message-ID: <20050210141217.87A971FF@aland.bbn.com> >I am writing a a networking library to facilitate faster data transfers using >UDP. The required level of performance is 120Mbps (at the minimum) on Gigabit >Ethernet. TCP should be able to do that without difficulty. Why use UDP??? >But at the moment what I am getting is around 32Mbps. I am using a 8kB packet >size and using Redhat 2.4.21 kernel (Enterperise Linux). I am having 4X >2.8GHz Intel Xeon CPUs with 4GB memory. Two identical machines are >interconnected using an SMC switch (Gigabit Ethernet) Not clear you've provided enough information here, particularly if you're send back acks for the UDP datagrams. Let's assume, for a moment, that this is a naive implementation and you're just sending UDP packets as fast as you can. You're sending or receiving a packet roughly every 65 us. How fast can your machine take interrupts? How fast can it copy memory through the processor? In short, what's your time budget and does it fit within 65us. (My rough estimate says it may be close -- 60ns DRAM with 16-byte chunks [128 bit bus] takes at least 30us to copy 8kB...) Craig From sahans at cse.mrt.ac.lk Thu Feb 10 07:22:54 2005 From: sahans at cse.mrt.ac.lk (sahans@cse.mrt.ac.lk) Date: Thu Feb 10 07:16:59 2005 Subject: [e2e] UDP Performance problem In-Reply-To: <20050210141217.87A971FF@aland.bbn.com> References: <20050210141217.87A971FF@aland.bbn.com> Message-ID: <1108048974.420b7c4e48411@mail.mrt.ac.lk> > TCP should be able to do that without difficulty. Why use UDP??? Final implementation will be multicast based one. We have higher data rates (transefer rate * no. of hosts) that point-to-point cannot achive even in 10Gbps infiniband. For the second issue .. I am just sending udp datagrams and and just increment a counter at the clientside on the recipt of data. I am not sure about the other parameters .. but I am having doubts about 10ms Linux timeslice ... Sahan Quoting Craig Partridge : > > >I am writing a a networking library to facilitate faster data transfers > using > >UDP. The required level of performance is 120Mbps (at the minimum) on > Gigabit > >Ethernet. > > TCP should be able to do that without difficulty. Why use UDP??? > > >But at the moment what I am getting is around 32Mbps. I am using a 8kB > packet > >size and using Redhat 2.4.21 kernel (Enterperise Linux). I am having 4X > >2.8GHz Intel Xeon CPUs with 4GB memory. Two identical machines are > >interconnected using an SMC switch (Gigabit Ethernet) > > Not clear you've provided enough information here, particularly if you're > send back acks for the UDP datagrams. Let's assume, for a moment, that > this is a naive implementation and you're just sending UDP packets as fast > as you can. > > You're sending or receiving a packet roughly every 65 us. How fast can > your machine take interrupts? How fast can it copy memory through the > processor? In short, what's your time budget and does it fit within > 65us. (My rough estimate says it may be close -- 60ns DRAM with 16-byte > chunks [128 bit bus] takes at least 30us to copy 8kB...) > > Craig > ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From craig at aland.bbn.com Thu Feb 10 07:19:15 2005 From: craig at aland.bbn.com (Craig Partridge) Date: Thu Feb 10 07:20:56 2005 Subject: clarification Re: [e2e] UDP Performance problem In-Reply-To: Your message of "Thu, 10 Feb 2005 09:12:17 EST." <20050210141217.87A971FF@aland.bbn.com> Message-ID: <20050210151915.8C432243@aland.bbn.com> Two clarifications/updates: 1. The 65us rate was dividing 1 Gbps by 8kB packets (not the reported 32Mbps, which when divided by 8kB packets translates to 512 packets per second, a data rate which would have been considered disturbingly low on your typical workstation c. 1988. 2. An colleague responded to my note by pointing out that on similar hardware he's seen 400 mb/s when downloading from the network to **DISK**. So there's something really sick in your system. Craig From kx2 at njit.edu Thu Feb 10 09:26:03 2005 From: kx2 at njit.edu (Roy Xu) Date: Thu Feb 10 09:27:21 2005 Subject: [e2e] link between Kelly's control and TCP's AIMD Message-ID: <000a01c50f95$98bfd660$f01aeb80@snow> Hi all, I'm looking for a pointer to literatures that link the TCP's (discrete) AIMD to Kelly's (continuous) control formulation. Many thanks in advance. --roy -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.postel.org/pipermail/end2end-interest/attachments/20050210/8a4aa79c/attachment.html From touch at ISI.EDU Thu Feb 10 10:23:04 2005 From: touch at ISI.EDU (Joe Touch) Date: Thu Feb 10 10:25:05 2005 Subject: [e2e] Spam filter problem being fixed Message-ID: <420BA688.3050007@isi.edu> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, all, You may have noticed some recent bouts of spam on this list. The spam filters have had some problems recently, and we're working to fix them as quickly as possible, FYI. Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (MingW32) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCC6aIE5f5cImnZrsRAutlAJ9Bf+2AKaxTo0cpIP3eqK0+gyb1GQCeOdon GytdbXxwWG5FOMA+6NJkI70= =7fKQ -----END PGP SIGNATURE----- From dovrolis at cc.gatech.edu Thu Feb 10 07:34:08 2005 From: dovrolis at cc.gatech.edu (Constantine Dovrolis) Date: Thu Feb 10 10:45:58 2005 Subject: [e2e] PAM'05: call for participation Message-ID: ================================================================ PAM 2005 http://www.pam2005.org/ Sixth international workshop on Passive and Active Measurement March 31 and April 1, 2005, Boston, MA CALL FOR PARTICIPATION ================================================================ Dear colleague, Enclosed you will find the advance program for the sixth international workshop on Passive and Active Measurement (PAM 2005) to be held in Boston, MA, on March 31 and April 1, 2005. The proceedings of the workshop will be published by Springer in the "Lecture Notes in Computer Science" (LNCS) series. The workshop is sponsored by Endace Measurement Systems and Intel. The advance registration deadline and the hotel reservation deadline is MARCH 1, 2005. You can find more information on how to register at: http://www.pam2005.org/registration.html --------------- Advance Program --------------- Thursday, March 31 2005 08:20 - 08:30 Opening Remarks (Mark Crovella, Chair) 08:30 - 09:45 Session 1: TCP Measurements * On the Impact of Bursting on TCP Performance Ethan Blanton (Purdue University), Mark Allman (International Computer Science Institute) * A Study of Burstiness in TCP Flows Srinivas Shakkottai (University of Illinois at Urbana-Champaign), Nevil Brownlee (University of Auckland) * On the Stationarity of TCP Bulk Data Transfers Guillaume URVOY-KELLER (Institut Eurecom) 09:45 - 10:20 Coffee Break 10:20 - 12:00 Session 2: Application Measurements * Toward the Accurate Identification of Network Applications Andrew Moore (University of Cambridge), Konstantina Papagiannaki (Intel Research Cambridge) * A Traffic Identification Method and Evaluations for a Pure P2P Application Satoshi Ohzahata (Tokyo University of Agriculture and Technology), Yoichi Hagiwara (Tokyo University of Agriculture and Technology), Matsuaki Terada (Tokyo University of Agriculture and Technology), Konosuke Kawashima (Tokyo University of Agriculture and Technology) * Analysis of Peer-to-Peer Traffic on ADSL Louis Plissonneau (France Telecom R&D), Jean-Laurent Costeux (France Telecom R&D), Patrick Brown (France Telecom R&D) * Analysis of Communities Of Interest in Data Networks Bill Aiello (AT&T Labs - Research), Chuck Kalmanek (AT&T Labs - Research), Patrick McDaniel (Pennsylvania State University), Shubho Sen (AT&T Labs - Research), Oliver Spatscheck (AT&T Labs - Research), Jacobus Van der Merwe (AT&T Labs - Research) 12:00 - 13:30 Lunch 13:30 - 15:10 Session 3: Network Inference and Problem Diagnosis * Binary vs Analogue Path Monitoring in IP Networks Hung Nguyen (Swiss Federal Institute of Technology- Lausanne (EPFL)), Patrick Thiran (Swiss Federal Institute of Technology- Lausanne (EPFL)) * Exploiting the IPID field to infer network path and end-system characteristics Weifeng Chen (University of Massachusetts at Amherst), Yong Huang (University of Massachusetts at Amherst), Bruno Ribeiro (University of Massachusetts at Amherst), Kyoungwon Suh (University of Massachusetts at Amherst), Honggang Zhang (University of Massachusetts at Amherst), Edmundo de Souza e Silva (Federal University of Rio de Janeiro), Jim Kurose (University of Massachusetts at Amherst), Don Towsley (University of Massachusetts at Amherst) * New Methods for Passive Estimation of TCP Round-Trip Times Bryan Veal (The University of Georgia), Kang Li (The University of Georgia), David Lowenthal (The University of Georgia) * Detecting Duplex Mismatch on Ethernet Stanislav Shalunov (Internet2), Richard Carlson (Internet2) 15:10 - 15:45 Coffee Break 15:45 - 17:30 Session 4: Poster Session * Traffic Classification using a Statistical Approach Denis Zuev (Intel Research Cambridge), Andrew Moore (University of Cambridge) * Self-learning IP Traffic Classification based on Statistical Flow Characteristics Sebastian Zander (Centre for Advanced Internet Architectures, Swinburne University of Technology), Thuy Nguyen (Centre for Advanced Internet Architectures, Swinburne University of Technology), Grenville Armitage (Centre for Advanced Internet Architectures, Swinburne University of Technology) * Measured Comparitive Performance of TCP Stacks Sam Jansen (University of Waikato), Anthony McGregor (University of Waikato and NLANR) * Applying Principles of Active Available Bandwidth Algorithms to Passive TCP Traces Marcia Zangrilli (College of William and Mary), Bruce Lowekamp (College of William and Mary) * A Network Processor Based Passive Measurement Node Ramaswamy Ramaswamy (Univ. of Massachusetts, Amherst), Ning Weng (Univ. of Massachusetts, Amherst), Tilman Wolf (Univ. of Massachusetts, Amherst) * A Merged Inline Measurement Method for Capacity and Available Bandwidth Le Thanh Man Cao (Osaka University), Go Hasegawa (Osaka University), Masayuki Murata (Osaka University) * Hopcount and End-to-End Delay: IPv6 Versus IPv4 Xiaoming Zhou (Delft University of Technology), Piet Van Mieghem (Delft University of Technology) * Scalable Coordination Techniques for Distributed Network Monitoring Manish Sharma (Boston University and Oracle Corporation), John Byers (Boston University) * Evaluating the Accuracy of Captured Snapshots by Peer-to-Peer Crawlers Daniel Stutzbach (University of Oregon), Reza Rejaie (University of Oregon) * HOTS: An OWAMP-Compliant Hardware Packet Timestamper Shu Zhang (NICT, Japan), Katsushi Kobayashi (NICT, Japan) * Practical Passive Lossy Link Inference Alexandros Batsakis (Johns Hopkins University), Tanu Malik (Johns Hopkins University), Andreas Terzis (Johns Hopkins University) * Merging Network Measurement with Data Transport Pavlos Papageorgiou (Department of Electrical and Computer Engineering, University of Maryland), Michael Hicks (Department of Computer Science, University of Maryland) Friday, April 1 2005 08:30 - 09:45 Session 5: Topology Measurements * Improved Algorithms for Network Topology Discovery Benoit Donnet (LiP6), Timur Friedman (LiP6), Mark Crovella (Boston University) * Using Simple Per-Hop Capacity Metrics to Discover Link Layer Network Topology Shane Alcock (University of Waikato), Anthony McGregor (University of Waikato), Richard Nelson (University of Waikato) * Revisiting Internet AS-level Topology Discovery Xenofontas Dimitropoulos (Georgia Tech), Dmitri Krioukov (CAIDA), Goerge Riley (Georgia Tech) 09:45 - 10:20 Coffee Break 10:20 - 12:00 Session 6: Wireless Network Measurements and Monitoring Facilities * Application, Network and Link Layer Measurements of Streaming Video over a Wireless Campus Network Jae Chung (Worcester Polytechnic Institute), Feng Li (Worcester Polytechnic Institute), Mingzhe Li (Worcester Polytechnic Institute), Huahui Wu (Worcester Polytechnic Institute), Mark Claypool (Worcester Polytechnic Institute), Robert Kinicki (Worcester Polytechnic Institute) * Measurement based analysis of the handover in a WLAN MIPv6 scenario Albert Cabellos-Aparicio (Universitat Politecnica de Catalunya), René Serral-Gracià (Universitat Politecnica de Catalunya), Loránd Jakab (Universitat Politecnica de Catalunya), Jordi Domingo-Pascual (Universitat Politecnica de Catalunya) * A Distributed Passive Measurement Infrastructure Patrik Carlsson (Blekinge Institute of Technology), Markus Fiedler (Blekinge Institute of Technology), Arne Nilsson (Blekinge Institute of Technology) * lambdaMON - A Passive Monitoring Facility for DWDM Optical Networks Joerg Micheel (NLANR/MNA) 12:00 - 13:30 Lunch 13:30 - 14:45 Session 7: Routing and Traffic Engineering Measurements * Internet Routing Policies and Round-Trip-Times Han Zheng (University of Cambridge, Computer Laboratory), Eng Keong Lua (University of Cambridge, Computer Laboratory), Marcelo Pias (University of Cambridge, Computer Laboratory), Timothy Griffin (Intel Research Cambridge) * Traffic Matrix Reloaded: Impact of Routing Changes Renata Teixeira (Univ. California San Diego), Nick Duffield (AT&T Labs -- Research), Jennifer Rexford (AT&T Labs -- Research), Matthew Roughan (Univ. of Adelaide) * Some Observations of Internet Stream Lifetimes Nevil Brownlee (The University of Auckland) 14:45 - 15:20 Coffee Break 15:20 - 16:45 Session 8: Spectroscopy and Bandwidth Estimation * Spectroscopy of traceroute delays Andre Broido (CAIDA), Young Hyun (CAIDA), kc claffy (CAIDA) * Measuring Bandwidth Between PlanetLab Host Sung-Ju Lee (HP Labs), Puneet Sharma (HP Labs), Sujata Banerjee (HP Labs), Sujoy Basu (HP Labs), Rodrigo Fonseca (UC Berkeley) * Comparison of Public End-to-end Bandwidth Estimation Tools on High-Speed Links Alok Shriram (CAIDA and University of North Carolina - Chapel Hill), Margaret Murray (CAIDA and TACC at University of Texas - Austin), Young Hyun (CAIDA), Nevil Brownlee (CAIDA and University of Auckland, NZ), Andre Broido (CAIDA), Marina Fomenkov (CAIDA), k claffy (CAIDA) From robertk at video-phones-evdo.com Thu Feb 10 12:07:30 2005 From: robertk at video-phones-evdo.com (Robert Kim, Wireless Internet Consultant) Date: Thu Feb 10 12:10:58 2005 Subject: [e2e] Spam filter problem being fixed In-Reply-To: <420BA688.3050007@isi.edu> Message-ID: Joe, Who is that guy whos email sends millions of bounce back canned messages? I'm new here an my first post was not pleasant. X---------------- Robert Kim, Wireless Internet Consultant, Wifi Hotspot Consultant Verizon Wireless Agent (TM) http://EVDO-Coverage.com Forum: http://secrets.wireless-internet-coverage.com/ Phones: http://video-phones-evdo.com2611 S Pacific Coast Highway 101 Cardiff by the Sea CA 92007 206 984 0880 "Internet Service Is ONLY Broadband with Broadband Customer Service"(tm) OUR QUEST: To Kill the Cubicle! (SM) ---Shalommmmmmmm------- ----------------;-)---- -----Original Message----- From: end2end-interest-bounces@postel.org [mailto:end2end-interest-bounces@postel.org] On Behalf Of Joe Touch Sent: Thursday, February 10, 2005 10:23 AM To: End-to-end Subject: [e2e] Spam filter problem being fixed -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 -- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.300 / Virus Database: 265.8.7 - Release Date: 2/10/2005 From arjuna at dcs.kcl.ac.uk Thu Feb 10 12:41:25 2005 From: arjuna at dcs.kcl.ac.uk (Arjuna Sathiaseelan) Date: Thu Feb 10 12:42:59 2005 Subject: [e2e] Question on XCP Message-ID: <62657.137.73.8.3.1108068085.squirrel@137.73.8.3> Dear All, I would be very much obliged if anyone could let me know how a XCP sender determines whether all the routers in its path is XCP enabled? The XCP sender works only if all the routers in its path are enabled with XCP. I have a mechanism similar to XCP which requires all the routers in its path to be enabled with the mechanism proposed by me. Thanks. Regds, Arjuna From falk at ISI.EDU Thu Feb 10 13:29:37 2005 From: falk at ISI.EDU (Aaron Falk) Date: Thu Feb 10 13:30:58 2005 Subject: [e2e] Question on XCP In-Reply-To: <62657.137.73.8.3.1108068085.squirrel@137.73.8.3> References: <62657.137.73.8.3.1108068085.squirrel@137.73.8.3> Message-ID: <20050210212933.GE520@isi.edu> Arjuna- The simplest, albeit imperfect, approach is to add a field in the packet header which each XCP-capable router would decrement. If the change in this field was less from the change in TTL, the endpoints would be aware that some routers were not practicing XCP. This is imperfect since tunnels would hid hops which might be practicing XCP and non-router queues might not be detectectable at all using this method. Strictly speaking, though, for XCP only the bottleneck router needs to be running the algorithm. Non-XCP queues which never congest need not participate. Of course, the trick is knowing which is which. :) You can find more on our XCP work at http://www.isi.edu/isi-xcp . Regards, --aaron Arjuna Sathiaseelan wrote: > > Dear All, > I would be very much obliged if anyone could let me know how a XCP > sender determines whether all the routers in its path is XCP > enabled? The XCP sender works only if all the routers in its path > are enabled with XCP. > > I have a mechanism similar to XCP which requires all the routers in > its path to be enabled with the mechanism proposed by me. Thanks. > Regds, > Arjuna From faber at ISI.EDU Thu Feb 10 14:16:26 2005 From: faber at ISI.EDU (Ted Faber) Date: Thu Feb 10 14:16:58 2005 Subject: [e2e] UDP Performance problem In-Reply-To: <1108048974.420b7c4e48411@mail.mrt.ac.lk> References: <20050210141217.87A971FF@aland.bbn.com> <1108048974.420b7c4e48411@mail.mrt.ac.lk> Message-ID: <20050210221626.GI13488@pun.isi.edu> On Thu, Feb 10, 2005 at 09:22:54PM +0600, sahans@cse.mrt.ac.lk wrote: > > TCP should be able to do that without difficulty. Why use UDP??? > Final implementation will be multicast based one. Hmmmm. You might not want to make multicast an add-on. High speed reliable delivery to one host and to many are considerably different problems. Good luck. -- Ted Faber http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.asc Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050210/9b98356d/attachment.bin From kartik at cs.fsu.edu Thu Feb 10 16:58:47 2005 From: kartik at cs.fsu.edu (Kartik Gopalan) Date: Thu Feb 10 17:03:24 2005 Subject: [e2e] Re: UDP Performance problem Message-ID: You might want to check if the particular model of your SMC switch supports jumbo frames in the first place -- most lower-end Gigabit switches in market don't. We get up 700 to 900Mbps with UDP on SMC8508T switches and Intel PRO/1000 NIC cards (depending on the interrupt coalescing value set on the NIC). My colleague has reported between 450 to 600Mbps with TCP. You may also need to pace the sending rate a little from your UDP application to achieve peak throughput, so that your NIC doesn't get overwhelmed and start randomly dropping packets. - Kartik > From: sahans@cse.mrt.ac.lk > Subject: [e2e] UDP Performance problem > To: end-to-end list > Message-ID: <1108030135.420b32b7399b1@mail.mrt.ac.lk> > Content-Type: text/plain; charset=ISO-8859-1 > Hi, > I am writing a a networking library to facilitate faster data transfers > using UDP. The required level of performance is 120Mbps (at the minimum) > on Gigabit Ethernet. But at the moment what I am getting is around > 32Mbps. I am using a 8kB packet size and using Redhat 2.4.21 kernel > (Enterperise Linux). I am having 4X 2.8GHz Intel Xeon CPUs with 4GB > memory. Two identical machines are interconnected using an SMC switch > (Gigabit Ethernet) Does anybody has an idea about this dissapoiniting > performance? > > > Thanks in advance > Sahan From sam.jansen at gmail.com Thu Feb 10 17:18:11 2005 From: sam.jansen at gmail.com (Sam Jansen) Date: Thu Feb 10 17:19:27 2005 Subject: [e2e] Windows XP TCP Behaviour Message-ID: <420C07D3.2020505@gmail.com> I have recently been doing some testing of TCP implementations on an isolated network we have available in our research group. I've noticed some odd behaviour from Windows XP Service Pack 2 and was wondering if anybody else had encountered such weirdness and whether it affects other versions of Windows. I've described the most notable problem at: http://www.wand.net.nz/~stj2/nsc/emu_winxp.html What we see here is that Windows is sending data outside of the receivers advertised window! I've also noticed Windows sending weird sized packets in what is supposed to be a simple bulk data transfer (often sending packets with a lot less than an MSS worth of data). Note that the situation described in that link is very simple; just two machines connected over an otherwise unused bottleneck link. -- Sam Jansen sam@wand.net.nz Wand Network Research Group http://www.wand.net.nz/~stj2 From Jon.Crowcroft at cl.cam.ac.uk Fri Feb 11 00:00:05 2005 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Fri Feb 11 00:01:05 2005 Subject: [e2e] Question on XCP In-Reply-To: Message from Aaron Falk of "Thu, 10 Feb 2005 13:29:37 PST." <20050210212933.GE520@isi.edu> Message-ID: In missive <20050210212933.GE520@isi.edu>, Aaron Falk typed: >>The simplest, albeit imperfect, approach is to add a field in the >>packet header which each XCP-capable router would decrement. If the >>change in this field was less from the change in TTL, the endpoints >>would be aware that some routers were not practicing XCP. This is >>imperfect since tunnels would hid hops which might be practicing XCP >>and non-router queues might not be detectectable at all using this >>method. if you start the xcp-router-capable-counter field at the same as ttl, then all you have to do is see if it == ttl at the receiver (or any intermediate node) to tell if you traversed an xcp-agnostic box. but: not sure about the TTL - lots of boxes might be layer 2 with some layer 3 function - we might if XCP tool off, consider implementing the intermediate node functionality in switches, which dont have to do ttl-- otherwise the receiver needs to know what the initial value of ttl was, which is non triv. (or reflect it) >>Strictly speaking, though, for XCP only the bottleneck router needs to >>be running the algorithm. Non-XCP queues which never congest need not >>participate. Of course, the trick is knowing which is which. :) Easier: If XCP has a function in the bottleneck router, then if you run XCP you can infer if that function is applied - you can see if there's packet loss without XCP operating, or ECN without But: If you could choose to deploy XCP in bottlenecks, and only in bottlenecks, you would probably have a good claim to being the Internet God. >>You can find more on our XCP work at http://www.isi.edu/isi-xcp . >> >>Regards, >> >>--aaron >> >> >> >>Arjuna Sathiaseelan wrote: >>> >>> Dear All, >>> I would be very much obliged if anyone could let me know how a XCP >>> sender determines whether all the routers in its path is XCP >>> enabled? The XCP sender works only if all the routers in its path >>> are enabled with XCP. >>> >>> I have a mechanism similar to XCP which requires all the routers in >>> its path to be enabled with the mechanism proposed by me. Thanks. >>> Regds, >>> Arjuna cheers jon From dpreed at reed.com Fri Feb 11 12:21:44 2005 From: dpreed at reed.com (David P. Reed) Date: Fri Feb 11 12:25:02 2005 Subject: [e2e] Question on XCP In-Reply-To: References: Message-ID: <420D13D8.1020404@reed.com> You could try to use path MTU discovery as a means to detect XCP capable things ... or fragmentation itself. From bj33.lee at samsung.com Fri Feb 11 06:49:39 2005 From: bj33.lee at samsung.com (Byoung-Joon Lee) Date: Fri Feb 11 15:21:37 2005 Subject: [e2e] CFP: ACM SIGCOMM Asia Workshop 2005 - Beijing, China Message-ID: <7376390.1108133366578.JavaMail.weblogic@ep_ml01> ** We sincerely apologize if you receive multiple copies of this announcement. Call for Participation ACM SIGCOMM Asia Workshop 2005 April 12 - 14, 2005 Beijing, China http://www.sigcomm.edu.cn Registration for ACM SIGCOMM Asia Workshop 2005 is now open! Advanced registration will be available till March 11, 2005. ---------------- Important Dates: ---------------- Early Registration ends March 11, 2005 Student Grant application ends Feb 25, 2005 Technical Program is Tuesday April 12 - Wednesday April 13 Tutorials are on Monday April 11 ----------------------------------------- Highlights of SIGCOMM Asia Workshop 2005: ----------------------------------------- ACM SIGCOMM Asia workshop 2005 provides an international technical forum for experts from industry and academia everywhere in the world, but especially from the Asia and pacific region, to exchange ideas and present results of ongoing research in next generation Internet with an emphasis on wireless and mobile networking areas. The workshop includes two tutorials and a technical program consisting of one invited talk and 24 refereed papers. The invited talk is by Fred Baker, Cisco The two tutorials are: *Adaptive Packet Scheduling in Wireless Networks by Victor Li, Hong Kong University *Virtual and Overlay Networks by Joe Touch, USC/ISI For more information about registration and the workshop, visit our website at http://www.sigcomm.edu.cn From e.w.vos at kpn.com Sat Feb 12 02:42:46 2005 From: e.w.vos at kpn.com (E.w.vos) Date: Sat Feb 12 02:43:05 2005 Subject: [e2e] {Filename?} You are made active Message-ID: An HTML attachment was scrubbed... URL: http://www.postel.org/pipermail/end2end-interest/attachments/20050212/47346162/attachment.html -------------- next part -------------- This is a message from the MailScanner E-Mail Virus Protection Service ---------------------------------------------------------------------- The original e-mail attachment "viupd02.cpl" has an unusual filename and could possibly be infected with a virus. As a precaution, the attachment has been quarantined. Virus scanner report for Sat Feb 12 02:42:53 2005: MailScanner: Control panel items are often used to hide viruses (viupd02.cpl) Quarantine location on the ISI-4-30-3 MailScanner: /var/spool/quarantine/20050212/j1CAgoQ28651 If you were expecting the attachment and would like to receive it, please forward this e-mail to action@isi.edu for assistance. If this is urgent, please call Action at x88289 after forwarding the message. Thank you, IPC Computing Services From chetansk at netd.com Mon Feb 14 21:23:19 2005 From: chetansk at netd.com (Chetan Kumar S) Date: Mon Feb 14 21:27:09 2005 Subject: clarification Re: [e2e] UDP Performance problem In-Reply-To: <20050210151915.8C432243@aland.bbn.com> References: <20050210151915.8C432243@aland.bbn.com> Message-ID: <42118747.1020209@netd.com> Craig Partridge wrote: >Two clarifications/updates: > > Just incase if U had not noticed: I hope there are no printf statements in the loop. A most common mistake by first timers Thanks Chetan S >1. The 65us rate was dividing 1 Gbps by 8kB packets (not the reported > 32Mbps, which when divided by 8kB packets translates to 512 packets > per second, a data rate which would have been considered disturbingly > low on your typical workstation c. 1988. > >2. An colleague responded to my note by pointing out that on similar > hardware he's seen 400 mb/s when downloading from the network to **DISK**. > >So there's something really sick in your system. > >Craig > > > From alokdube at hotpop.com Tue Feb 15 02:27:48 2005 From: alokdube at hotpop.com (Alok) Date: Tue Feb 15 02:29:11 2005 Subject: [e2e] purpose of pseudo header in TCP checksum Message-ID: <006d01c51349$038a1ad0$0e0218ac@rs.riverstonenet.com> Hi, As per RFC 793, This pseudo header contains the Source Address, the Destination Address, the Protocol, and TCP length. This gives the TCP protection against misrouted segments. This information is carried in the Internet Protocol and is transferred across the TCP/Network interface in the arguments or results of calls by the TCP on the IP. +--------+--------+--------+--------+ | Source Address | +--------+--------+--------+--------+ | Destination Address | +--------+--------+--------+--------+ | zero | PTCL | TCP Length | +--------+--------+--------+--------+ The TCP Length is the TCP header length plus the data length in octets (this is not an explicitly transmitted quantity, but is computed), and it does not count the 12 octets of the pseudo header. what is the purpose of the using SA and DA for checksum calculations for the TCP header? Is it because that the SA/DA may be corrupted as the IP header checksum does not run on SA/DA ? I mean I dont understand what "This gives the TCP protection against misrouted segments" means, do intermediate routing nodes look at TCP checksum? -thanks Alok From dpreed at reed.com Tue Feb 15 04:39:39 2005 From: dpreed at reed.com (David P. Reed) Date: Tue Feb 15 04:41:13 2005 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <006d01c51349$038a1ad0$0e0218ac@rs.riverstonenet.com> References: <006d01c51349$038a1ad0$0e0218ac@rs.riverstonenet.com> Message-ID: <4211ED8B.1070301@reed.com> As I was there (in 1976, when we split TCP into IP, TCP, and other protocols, such as UDP) for the decision to separate the checksums and to create a pseudo-header, here is the rationale, which is highly relevant. TCP (and UDP) are end-to-end protocols. In particular, the TCP checksum is "end-to-end". It is a "private matter" between end points implementing the TCP layer, guaranteeing end-to-end reliability, not hop-by-hop reliability. IP is a wrapper for TCP, which instructs the transport layer (the gateways and routers) where the packet is to be transported, how big it is, and how it may be fragmented in the process of delivery.. The Source Address, Destination address, length, etc. are part of the meaning of the TCP frame - in that the end point machines use that information in the TCP application. Thus the function of SA, DA, etc. are "shared" because they are meaningful to both layers (IP and TCP). Rather than include the same information twice in the packet format, the concept of a "virtual header" was invented to encapsulate the idea that IP is not allowed to change the SA and DA because they are meaningful. Further, in the case of end-to-end encryption (in 1976 we had a complete design by Steven T. Kent, my office mate, which was blocked by NSA from being deployed) it is essential that all end-to-end meaning be protected. The plan was to leave the SA and DA in the clear, but encrypt the rest of the TCP payload, including the checksum. This would protect against a man-n-the-middle attack that delivered valid packets with an incorrect source address, etc. (yes, to be truly reliable, we would have had to use a DSA instead of the current checksum). This was a careful design decision, wrecked irrevocably by the terrorists who invented NAT (which doesn't allow end--to-end encryption, because NAT is inherently a "man-in-the-middle" attack!). The rise of the middleboxen have now so thoroughly corrupted the Internet protocol design that it's not surprising that the original designs are difficult to decode. If we actually had end-to-end encrypted TCP (now impossible because of the NATs) we would have a much more secure and safe Internet, while preserving its open character. Instead we have a maze of twisty, disconnected passages, vulnerable to a zillion hackers. From jnc at mercury.lcs.mit.edu Tue Feb 15 06:35:59 2005 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Tue Feb 15 06:37:12 2005 Subject: [e2e] purpose of pseudo header in TCP checksum Message-ID: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> > From: "Alok" > I dont understand what "This gives the TCP protection against > misrouted segments" means, do intermediate routing nodes look at > TCP checksum? What it means is that a software/hardware bug just about anywhere in the network might mean that a packet intended for host X arrives at host Y (e.g. by overwriting the destination address X with Y). If the SA/DA were not included in the TCP checksum, such a packet would pass the TCP checksum test. > Is it because that the SA/DA may be corrupted as the IP header > checksum does not run on SA/DA ? In IPv4, the IP header checksum does cover the SA/DA - it covers everything in the IP header, including the hop count. In IPv6, there is no header checksum, of course. Noel From jnc at mercury.lcs.mit.edu Tue Feb 15 06:58:22 2005 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Tue Feb 15 06:59:52 2005 Subject: [e2e] purpose of pseudo header in TCP checksum Message-ID: <20050215145822.36724872CF@mercury.lcs.mit.edu> > From: "David P. Reed" This is kind of orthagonal to the original poster's point, but I see an incorrect assertion I want to point out. > wrecked irrevocably by the terrorists who invented NAT Passing note: this may well set a new mark for corruption of the term "terrorist". (I concede that you may well be simply making fun of others who are doing the same thing.) > (which doesn't allow end-to-end encryption, because NAT is > inherently a "man-in-the-middle" attack!). > .. > If we actually had end-to-end encrypted TCP (now impossible because > of the NATs) David, as a general statement of what is and is not possible in a world with NAT boxes (defined as "internetwork level addresses are local in scope, and such addresses in internetwork level headers get modified crossing scope boundaries"), this is just not so. Yes, the *existing* secure TCP won't work across NAT boxes. However, it is not that difficult to design a TCP upgrade which a) runs across NAT boxes without needing to have the NAT boxes tweak the packets, and b) can be end-end secured (either authenticated, or privacy-protected; in both cases covering MiM attacks too). I know this because as part of the NSRG work I actually did most of such a design, which you can find here: http://ana-3.lcs.mit.edu/~jnc/tech/nsrg/tcp_doc.txt Briefly, if we got off our butts and added another namespace to the overall architecture, one intended for end-end naming (instead of trying to leech off the internetwork layer's routing names - which was pointed out as a mistake by Jerry Saltzer over 20 years ago), and changed TCP to use that in the pseudo-header, NAT problems become non-existent (except when doing an ICP to a legacy host). If we stopped chasing the worthless chimera of IPv6, maybe some useful things like the above would actually get done. I'm not holding my breath on that one. > Instead we have a maze of twisty, disconnected passages, vulnerable > to a zillion hackers. One day people will realize that the only real solution to security in an open network environment (given all the buggy applications out there) is to run applications in AIM boxes. I'm not holding my breath on that one either. Noel From michael.welzl at uibk.ac.at Tue Feb 15 08:05:06 2005 From: michael.welzl at uibk.ac.at (Michael Welzl) Date: Tue Feb 15 08:11:11 2005 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> References: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> Message-ID: <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> > In IPv4, the IP header checksum does cover the SA/DA - it covers > everything in the IP header, including the hop count. In IPv6, there is > no header checksum, of course. ...which was a bad decision IMO, because it breaks the "IP over everything" model and will now lead to problems with mechanisms such as UDP Lite, where link layers are now required check the headers (but ONLY the headers) of things above. Yuck. Cheers, Michael From dpreed at reed.com Tue Feb 15 08:11:38 2005 From: dpreed at reed.com (David P. Reed) Date: Tue Feb 15 08:13:11 2005 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <20050215145822.36724872CF@mercury.lcs.mit.edu> References: <20050215145822.36724872CF@mercury.lcs.mit.edu> Message-ID: <42121F3A.1030508@reed.com> Noel - you are correct, and I agree that it is better. If we were to view the IP namespace as merely a routing "hint" and put the actual end point address in the TCP layer, end-to-end that would indeed solve the problem. I also agree that IPv6 continues to perpetuate this confusion of "shared fields" between the end-to-end protocol and the IP layer. Your approach is cleaner, and I wish we had had the guts to push that through in the 1976 process - despite the bias toward character echoing that pervaded a large, pragmatically motivated part of the design community at the time. (and who knows, had we done it "right" we might have never had enough adoption for the correct design to survive). I stand by my hyperbolic statement about "terrorists" - you may recall that the original sales of NAT boxes claimed that NAT was an Internet "Standard" when it was merely part of a set of alternatives being considered for a next generation solution to the address space shortage. Those who deceived the world by marketing this non-standard as if it were an accepted IETF standard were indeed creating havoc that has compromised the network irrevocably. IMHO, of course. :-) I wrote about it at the time... as did Larry Lessig. From dpreed at reed.com Tue Feb 15 08:35:22 2005 From: dpreed at reed.com (David P. Reed) Date: Tue Feb 15 08:37:12 2005 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: References: <006d01c51349$038a1ad0$0e0218ac@rs.riverstonenet.com> <4211ED8B.1070301@reed.com> Message-ID: <421224CA.8050406@reed.com> Lloyd - you're right, I'm guilty of judging intent here. And being sanctimonious on this topic particularly... but the "twit" is a bit trite... I'm sure that the NAT box marketers thought of themselves as "freedom fighters", given the attempt by access providers to control the number of routable devices in the home as a way to boost their profits by slowing innovation. However, NAT boxes did externalize a cost onto the entire Internet community, and did so by pretending that NAT boxes were compatible with the Internet architecture's design for scalability. However, somebody has to point out that the "freedom fighters" have hands that are not so clean of collateral damage, which is exactly why I label them "terrorists". We could have a side discussion about the moral culpability of governments who ruthlessly attack civilian populations in order to hold them accountable for terrorists who live among them... who are the terrorists? The word "terrorist" is so meaningless that we might as well use it to describe ADD children. From alokdube at hotpop.com Tue Feb 15 08:45:29 2005 From: alokdube at hotpop.com (Alok) Date: Tue Feb 15 08:47:44 2005 Subject: [e2e] purpose of pseudo header in TCP checksum References: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> Message-ID: <000801c5137d$c74e14b0$0e0218ac@rs.riverstonenet.com> not really, it was a design decision given the constraints :) good and bad are for time to decide........... ----- Original Message ----- From: "Michael Welzl" To: "Noel Chiappa" Cc: ; Sent: Tuesday, February 15, 2005 9:35 PM Subject: Re: [e2e] purpose of pseudo header in TCP checksum > > In IPv4, the IP header checksum does cover the SA/DA - it covers > > everything in the IP header, including the hop count. In IPv6, there is > > no header checksum, of course. > > ...which was a bad decision IMO, because it breaks the > "IP over everything" model and will now lead to problems > with mechanisms such as UDP Lite, where link layers are > now required check the headers (but ONLY the headers) > of things above. Yuck. > > Cheers, > Michael > From touch at ISI.EDU Tue Feb 15 09:38:33 2005 From: touch at ISI.EDU (Joe Touch) Date: Tue Feb 15 09:39:39 2005 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> References: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> Message-ID: <42123399.7050508@isi.edu> Michael Welzl wrote: >>In IPv4, the IP header checksum does cover the SA/DA - it covers >>everything in the IP header, including the hop count. In IPv6, there is >>no header checksum, of course. > > ...which was a bad decision IMO, because it breaks the > "IP over everything" model and will now lead to problems > with mechanisms such as UDP Lite, where link layers are > now required check the headers (but ONLY the headers) > of things above. Yuck. IPv6 _requires_ UDP checksums. UDP Lite already addresses this, and when used ALWAYS INCLUDES the pseudoheader. See RFC 3828. IP still works over everything; things that work over IP need to know something about it, as always. Joe -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 254 bytes Desc: OpenPGP digital signature Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050215/77118761/signature.bin From michael.welzl at uibk.ac.at Tue Feb 15 09:46:20 2005 From: michael.welzl at uibk.ac.at (Michael Welzl) Date: Tue Feb 15 09:51:12 2005 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <42123399.7050508@isi.edu> References: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> <42123399.7050508@isi.edu> Message-ID: <1108489580.4768.228.camel@lap10-c703.uibk.ac.at> > IPv6 _requires_ UDP checksums. UDP Lite already addresses this, and when > used ALWAYS INCLUDES the pseudoheader. See RFC 3828. I know that checking the pseudoheader is required with IPv6 - but only end systems do this, right? The part I don't get is about intermediate systems; it wouldn't surprise me if it's just a misunderstanding on my part. What I mean is: since the IPv6 header isn't checked by routers anymore, there must be some assumption that link layers check it, or an intermediate router might (for instance) be sending packets to god knows where. Now, with UDP Lite, we DON'T want link layer to do this check. But wait, if we simply say "don't", we'll have lots of problems - first, because Adler-32 just isn't good enough, and second, because IPv6 doesn't check it, and once again, a router might wind up making decisions based on an erroneous packet header. So we need to specify that link layers DO check the headers and just do NOT check the payload if it's a UDP Lite packet. This is the part that I consider inevitable (given the current situation) but ugly. > IP still works over everything; things that work over IP need to know > something about it, as always. "over" is fine with me, but I don't like to have things underneath know so much about it. Cheers, Michael From weddy at grc.nasa.gov Tue Feb 15 10:19:44 2005 From: weddy at grc.nasa.gov (Wesley Eddy) Date: Tue Feb 15 10:25:15 2005 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <1108489580.4768.228.camel@lap10-c703.uibk.ac.at> References: <42123399.7050508@isi.edu> <1108489580.4768.228.camel@lap10-c703 .uibk.ac.at> Message-ID: <20050215181944.GC21245@grc.nasa.gov> On Tue, Feb 15, 2005 at 06:46:20PM +0100, Michael Welzl wrote: > > So we need to specify that link layers DO check the headers > and just do NOT check the payload if it's a UDP Lite packet. > This is the part that I consider inevitable (given the > current situation) but ugly. > Actually, with UDP-Lite, you'd be asking the link layers to parse up through the transport layer and check as much of the data as the UDP-Lite header designates as sensitive. That's slightly more complex than just checking the protocol field in IP header, which already assumes that the link device can parse IP, can see through whatever levels of tunnels are present, etc. The reasonableness of the architecture that UDP-Lite would like to operate in seems rather questionable to me. -Wes -- Wesley M. Eddy NASA GRC / Verizon FNS http://roland.grc.nasa.gov/~weddy From lln at csee.ltu.se Tue Feb 15 11:24:51 2005 From: lln at csee.ltu.se (=?ISO-8859-1?Q?Lars-=C5ke_Larzon?=) Date: Tue Feb 15 11:25:23 2005 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <20050215181944.GC21245@grc.nasa.gov> References: <42123399.7050508@isi.edu> <1108489580.4768.228.camel@lap10-c703 .uibk.ac.at> <20050215181944.GC21245@grc.nasa.gov> Message-ID: <42124C83.1000201@csee.ltu.se> Wesley Eddy wrote: > Actually, with UDP-Lite, you'd be asking the link layers to parse up > through the transport layer and check as much of the data as the > UDP-Lite header designates as sensitive. That's slightly more complex > than just checking the protocol field in IP header, which already > assumes that the link device can parse IP, can see through whatever > levels of tunnels are present, etc. We're getting slightly off topic here, but just to clarify about UDP-Lite: UDP-Lite gives you *the option* to use the coverage field as an indication of how much data to checksum in the link layer if you have a link layer supporting that kind of behavior. As it turns out, many of the scenarios where this might be desirable have a significantly higher cost for retransmission compared to the extra parsing that needs to be done. The recommended default behavior of UDP-Lite, though, is to work exactly as UDP, i.e., to checksum the entire datagram. The partial checksum is entirely optional and it is up to application and system designers to decide whether it is worth it or not. /Lars-?ke -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3224 bytes Desc: S/MIME Cryptographic Signature Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050215/c601d2b1/smime.bin From touch at ISI.EDU Tue Feb 15 11:24:49 2005 From: touch at ISI.EDU (Joe Touch) Date: Tue Feb 15 11:25:55 2005 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <1108489580.4768.228.camel@lap10-c703.uibk.ac.at> References: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> <42123399.7050508@isi.edu> <1108489580.4768.228.camel@lap10-c703.uibk.ac.at> Message-ID: <42124C81.1000902@isi.edu> Michael Welzl wrote: >>IPv6 _requires_ UDP checksums. UDP Lite already addresses this, and when >>used ALWAYS INCLUDES the pseudoheader. See RFC 3828. > > I know that checking the pseudoheader is required with IPv6 - > but only end systems do this, right? Yes - that's a UDP step. > The part I don't get is > about intermediate systems; it wouldn't surprise me if it's just > a misunderstanding on my part. > > What I mean is: since the IPv6 header isn't checked by routers > anymore, there must be some assumption that link layers > check it, or an intermediate router might (for instance) > be sending packets to god knows where. IPv6 makes no mention of a problem with 'sending packets god knows where', so long as they're checked at the receiver. Although IPv6 (at least as I recall) omits the IP checksum because of sufficient link checksums to detect corrupted packets, there's no requirement that link layers have such checksums (as hinted in the UDP-Lite RFC) - at least not in 2460 (anyone else know anywhere else a body is buried in this regard?) FWIW, IPv6 adds a checksum to ICMP, again to prevent corrupted packets from being received, but there's not apparent concern for misforwarding nased on corruption. > Now, with UDP Lite, we DON'T want link layer to do this > check. But wait, if we simply say "don't", we'll have > lots of problems - first, because Adler-32 just isn't > good enough, and second, because IPv6 doesn't check it, > and once again, a router might wind up making decisions > based on an erroneous packet header. I guess IPv6 says "so"? Even IPsec, required by IPv6, works primary at 'endpoints' (where tunnel endpoints are effectively end hosts for the purposes of architecture). > So we need to specify that link layers DO check the headers > and just do NOT check the payload if it's a UDP Lite packet. > This is the part that I consider inevitable (given the > current situation) but ugly. The first part, not AFAICT. The second IS ugly. Complain to UDP-Lite - it's definitely reaching across layers to coordinate the link checksum configuration with UDP's presence or lack of checksumming. >>IP still works over everything; things that work over IP need to know >>something about it, as always. > > "over" is fine with me, but I don't like to have things > underneath know so much about it. Agreeed. IMO, that's UDP-Lite's problem, not IPv6. Joe -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 254 bytes Desc: OpenPGP digital signature Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050215/4ddbd251/signature.bin From michael.welzl at uibk.ac.at Tue Feb 15 11:42:03 2005 From: michael.welzl at uibk.ac.at (Michael Welzl) Date: Tue Feb 15 11:47:16 2005 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <42124C81.1000902@isi.edu> References: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> <42123399.7050508@isi.edu> <1108489580.4768.228.camel@lap10-c703.uibk.ac.at> <42124C81.1000902@isi.edu> Message-ID: <1108496523.4768.244.camel@lap10-c703.uibk.ac.at> > IPv6 makes no mention of a problem with 'sending packets god knows > where', so long as they're checked at the receiver. Although IPv6 (at > least as I recall) omits the IP checksum because of sufficient link > checksums to detect corrupted packets, there's no requirement that link > layers have such checksums (as hinted in the UDP-Lite RFC) - at least > not in 2460 (anyone else know anywhere else a body is buried in this > regard?) I also searched RFC 2460 for this once, but couldn't find anything. Still, that doesn't make sense to me. What if you transfer an IPv6 packet across a link layer that does *NOT* check your data - why wouldn't this cause harm to the source / destination addresses, and therefore cause wrong router behavior downstream? And what about the other fields - say, "next header"; a checksum error could turn a TCP packet into a UDP packet, change the payload length field, whatever. There must be a requirement for link layers to do a strong check somewhere?! > > So we need to specify that link layers DO check the headers > > and just do NOT check the payload if it's a UDP Lite packet. > > This is the part that I consider inevitable (given the > > current situation) but ugly. > > The first part, not AFAICT. > > The second IS ugly. Complain to UDP-Lite - it's definitely reaching > across layers to coordinate the link checksum configuration with UDP's > presence or lack of checksumming. > > >>IP still works over everything; things that work over IP need to know > >>something about it, as always. > > > > "over" is fine with me, but I don't like to have things > > underneath know so much about it. > > Agreeed. IMO, that's UDP-Lite's problem, not IPv6. What if we had a CRC-32 checksum in the IPv4 header, in the IPv6 header, in UDP Lite's header, in TCP's header, in the UDP header, well - just all over the place. Then, we could just say "disable your checksums" to link layers when they notice a UDP Lite packet. It would even make sense to have a "corruption acceptable" bit somewhere in the IP header in such a scenario (I know, no bits to spare). Wouldn't that be better design? Cheers, Michael From anoop.ghanwani at hp.com Tue Feb 15 13:37:09 2005 From: anoop.ghanwani at hp.com (Ghanwani, Anoop) Date: Tue Feb 15 13:39:13 2005 Subject: [e2e] purpose of pseudo header in TCP checksum Message-ID: <83AB0942FD087D499DF2DD5CEE1B6133013CC353@cacexc06.americas.cpqcorp.net> > I also searched RFC 2460 for this once, but couldn't find anything. > Still, that doesn't make sense to me. What if you transfer an > IPv6 packet across a link layer that does *NOT* check your data - > why wouldn't this cause harm to the source / destination addresses, > and therefore cause wrong router behavior downstream? And > what about the other fields - say, "next header"; a checksum > error could turn a TCP packet into a UDP packet, change the > payload length field, whatever. > > There must be a requirement for link layers to do a strong > check somewhere?! This is something that struck me as really odd as well when I first started working on IPv6. Even if the link layer does provide a good CRC, what happens if the IPv6 header gets corrupted while the packet is being processed in a router and it doesn't have a link layer header on it? This seems to imply that routers are required to generate checksums on the IP packet for internal use in order to guarantee the integrity of the IP header as the packet is processed. If this is not done, IPv6 packets could end up far away from the source of the problem (and far away from their intended destination) making things really hard to debug! Anoop From touch at ISI.EDU Tue Feb 15 15:20:14 2005 From: touch at ISI.EDU (Joe Touch) Date: Tue, 15 Feb 2005 15:20:14 -0800 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <1108496523.4768.244.camel@lap10-c703.uibk.ac.at> References: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> <42123399.7050508@isi.edu> <1108489580.4768.228.camel@lap10-c703.uibk.ac.at> <42124C81.1000902@isi.edu> <1108496523.4768.244.camel@lap10-c703.uibk.ac.at> Message-ID: <421283AE.6020508@isi.edu> Michael Welzl wrote: >>IPv6 makes no mention of a problem with 'sending packets god knows >>where', so long as they're checked at the receiver. Although IPv6 (at >>least as I recall) omits the IP checksum because of sufficient link >>checksums to detect corrupted packets, there's no requirement that link >>layers have such checksums (as hinted in the UDP-Lite RFC) - at least >>not in 2460 (anyone else know anywhere else a body is buried in this >>regard?) > > > I also searched RFC 2460 for this once, but couldn't find anything. > Still, that doesn't make sense to me. What if you transfer an > IPv6 packet across a link layer that does *NOT* check your data - > why wouldn't this cause harm to the source / destination addresses, > and therefore cause wrong router behavior downstream? It doesn't cause harm at the destination, since there's supposed to be a transport checksum (even for ICMP - one was added). I suppose there's no perceived harm in sending some corrupted traffic through the wrong paths. Note - this is designed to address corruption, not malicious behavior. > And > what about the other fields - say, "next header"; a checksum > error could turn a TCP packet into a UDP packet, change the > payload length field, whatever. Sure - but then that checksum should either match or somebody's overwriting packets anyway, which they can do even if there are IP checksums. Onec you overwrite things, all bets are off. > There must be a requirement for link layers to do a strong > check somewhere?! Link won't find routers that rewrite headers ;-) And IP is supposed to work over everything - not all link layers have checksums. >>>So we need to specify that link layers DO check the headers >>>and just do NOT check the payload if it's a UDP Lite packet. >>>This is the part that I consider inevitable (given the >>>current situation) but ugly. >> >>The first part, not AFAICT. >> >>The second IS ugly. Complain to UDP-Lite - it's definitely reaching >>across layers to coordinate the link checksum configuration with UDP's >>presence or lack of checksumming. >> >> >>>>IP still works over everything; things that work over IP need to know >>>>something about it, as always. >>> >>>"over" is fine with me, but I don't like to have things >>>underneath know so much about it. >> >>Agreeed. IMO, that's UDP-Lite's problem, not IPv6. > > What if we had a CRC-32 checksum in the IPv4 header, in > the IPv6 header, in UDP Lite's header, in TCP's header, > in the UDP header, well - just all over the place. The argument (far as I recall) was that things would be slower, not better. > Then, we could just say "disable your checksums" to > link layers when they notice a UDP Lite packet. It would > even make sense to have a "corruption acceptable" bit > somewhere in the IP header in such a scenario (I know, > no bits to spare). > > Wouldn't that be better design? Define better. Again, the assumption is that not that many packets are corrupted, and that when they are their path doesn't matter so long as the packets aren't accepted at the alleged destination. How would checksums everywhere help that? Is it necessary (and worth it) to do more (the argument for IPv6 was "no")? Joe -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 254 bytes Desc: OpenPGP digital signature Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050215/dc4590ad/signature.bin From touch at ISI.EDU Tue Feb 15 15:20:12 2005 From: touch at ISI.EDU (Joe Touch) Date: Tue, 15 Feb 2005 15:20:12 -0800 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <83AB0942FD087D499DF2DD5CEE1B6133013CC353@cacexc06.americas.cpqcorp.net> References: <83AB0942FD087D499DF2DD5CEE1B6133013CC353@cacexc06.americas.cpqcorp.net> Message-ID: <421283AC.3060801@isi.edu> Ghanwani, Anoop wrote: >>I also searched RFC 2460 for this once, but couldn't find anything. >>Still, that doesn't make sense to me. What if you transfer an >>IPv6 packet across a link layer that does *NOT* check your data - >>why wouldn't this cause harm to the source / destination addresses, >>and therefore cause wrong router behavior downstream? And >>what about the other fields - say, "next header"; a checksum >>error could turn a TCP packet into a UDP packet, change the >>payload length field, whatever. >> >>There must be a requirement for link layers to do a strong >>check somewhere?! > > This is something that struck me as really odd as well when > I first started working on IPv6. Even if the link layer does > provide a good CRC, what happens if the IPv6 header gets corrupted > while the packet is being processed in a router and it doesn't > have a link layer header on it? > > This seems to imply that routers are required to generate > checksums on the IP packet for internal use in order to > guarantee the integrity of the IP header as the packet is > processed. If this is not done, IPv6 packets could end up > far away from the source of the problem (and far away from > their intended destination) making things really hard to > debug! And if the bugs are more likely to be in the router code than on the wire (I recall a few people noticing this), then using checksums to check for bugs in the code that might have bugs that fail to check the checksums is circularly dubious. Joe -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 254 bytes Desc: OpenPGP digital signature Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050215/1c31205a/signature.bin From avg at kotovnik.com Tue Feb 15 18:20:20 2005 From: avg at kotovnik.com (Vadim Antonov) Date: Tue, 15 Feb 2005 18:20:20 -0800 (PST) Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <20050215145822.36724872CF@mercury.lcs.mit.edu> Message-ID: The pseudo-headers, essentially, guard against an end-system implementation which would accept packets not addressed to it. This is just one of a zillion ways in which any realistic OS can be broken to the effect that it'll corrupt data. And, given the simplicity of comparing destination address on a packet to the list of host's addresses, it is a quite unlikely bug. On the other hand, pseudo-headers restrict the implementation of the packet layer by enforcing some form of an address (of course, they can be computed over 4-byte hashes of the actual addresses, but this is not how it is specified). Simply put, this is a non-solution to a non-existing problem, at the cost of an unnecessary inter-layer dependency. A useful security cannot be implemented at IP or TCP layer, because security is only as good as its weakest point. Any strong security requires entity authentication (which means, pretty much, authenticating a physical user, a server process, and determination of trustworthiness of an execution environment), access authorization (meaning determination of which entity is allowed to do what), privacy and integrity protection (encryption and message authentication), intrusion detection and reaction capabilities, and key managament. Network layer security is not aware of users or servers, cannot do access authorization on any useful granularity level, and does not do much for intrusion detection and reaction. It follows that it is quite weak. I wouldn't call IPsec, SSL/TLS, etc, "secure" - they can be used as rather trivial parts of much larger security architectures, but they provide only false sense of security when deployed stand-alone. These weaknesses necessiate the use of manually-configured firewalls and other kludges, which are nice and dandy if you only want protection against script kiddies. However, most attacks doing any significant damage come from inside of security perimeters. Again, IPsec and all those "secure" VPNs do, essentially, nothing to protect against those attacks, and many sysadmins discovered that their salesdroids' "secure" laptops with "secure" VPN connections to the office get infected with virii downloaded from the Internet by means of unprotected home DSL connections and happily proceed to spread the infection all over their "secure" LANs with quite depressing regularity. Essentially, the only way to build a reasonably secure system, having an elusive but quite important property called "compartmentalization" is to do it at the application/middleware level, and treat networks as always insecure. This also means that attempts to drag security down to the network layer are, at best, misguided. The real problem with NAT traversal by encrypted comms is not in the fact that some broken-as-desiged protocols don't work; there are ones which do work fine no matter how you chop & mix the bytestream in the intermediate systems. The problem is in the application-level protocols which embed transport-level addresses (notoriously FTP, and most of the VoIP signaling crap). There is no excuse for doing that. The only place where such embedding is unavoidable is the name resolution protocol - and it does not need to be encrypted, since end-host authentication is required anyway to protect against man-in-the-middle attacks. A useful transport-level design would provide a special zone in the packets for clear-text transport addresses which would be understood and translated by NATs as packets go through. On Tue, 15 Feb 2005, Noel Chiappa wrote: > Briefly, if we got off our butts and added another namespace to the > overall architecture, one intended for end-end naming (instead of trying > to leech off the internetwork layer's routing names - which was pointed > out as a mistake by Jerry Saltzer over 20 years ago), and changed TCP to > use that in the pseudo-header, NAT problems become non-existent (except > when doing an ICP to a legacy host). I'm not sure that an extra level of indirection is good, as things can be done with just two levels (names and routing addresses) if network services register & deregister themselves dynamically with the name servce. The way I see it, there is no need to have the fixed host (port) / network (IP address) boundary at all (the fixed network/host address boundaries are already history). There's even no need for strong consistency in the distributed name caching - if your connection to the address failed with "no such end-point" error received from the remote host, you can simply tell your name resolver to invalidate cache entries and request the up-to-date information from the source. It helps if addresses (including ports) are never reused faster than name serivce cache entries expire. Fixed port numbers for WKSes are a kludge, and deserve to die (the leftmost component of a domain name can be always treated as the service name, as people apt to do anyway, with ubiquitous www.blah.com-s and mail.blah.com-s). The dynamic service registration takes care of renumbering, too, thus making address block ownership a non-issue. It is not hard to implement, and is, in fact, much simpler than the existing kludge^H^H^H^H^H^HDNS infrastructure. --vadim From michael.welzl at uibk.ac.at Wed Feb 16 01:31:53 2005 From: michael.welzl at uibk.ac.at (Michael Welzl) Date: 16 Feb 2005 10:31:53 +0100 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <421283AE.6020508@isi.edu> References: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> <42123399.7050508@isi.edu> <1108489580.4768.228.camel@lap10-c703.uibk.ac.at> <42124C81.1000902@isi.edu> <1108496523.4768.244.camel@lap10-c703.uibk.ac.at> <421283AE.6020508@isi.edu> Message-ID: <1108546313.4764.160.camel@lap10-c703.uibk.ac.at> > > I also searched RFC 2460 for this once, but couldn't find anything. > > Still, that doesn't make sense to me. What if you transfer an > > IPv6 packet across a link layer that does *NOT* check your data - > > why wouldn't this cause harm to the source / destination addresses, > > and therefore cause wrong router behavior downstream? > > It doesn't cause harm at the destination, since there's supposed to be a > transport checksum (even for ICMP - one was added). I suppose there's no > perceived harm in sending some corrupted traffic through the wrong paths. > > Note - this is designed to address corruption, not malicious behavior. Okay, I agree that end-to-end integrity is preserved, nothing wrong about that. It's just that I find it hard to believe that there's no perceived harm in having routers make decisions based upon an erroneous IP header. Is sending some corrupted traffic through the wrong paths really all that can happen? Hm, it could affect the hop limit, too, but who cares. Okay, so what about the "payload length" field - couldn't it lead to errors (a router allocates a buffer that is way too large ... hmmm, do we care?) ... I'm unsure, but still, I find it hard to believe. > And > > what about the other fields - say, "next header"; a checksum > > error could turn a TCP packet into a UDP packet, change the > > payload length field, whatever. > > Sure - but then that checksum should either match or somebody's > overwriting packets anyway, which they can do even if there are IP > checksums. Onec you overwrite things, all bets are off. I don't get that. There's no IPv6 checksum, so these fields could be changed by corruption and noone would notice. We could say that we don't care about the "next header" field too, because that's relevant in the end system, where the pseudo header will take effect. So, I wonder: was the reason not to have an IPv6 checksum: * because the assumption was that link layers provide strong CRC protection anyway ( = "IP no longer over everything") or * because end-to-end integrity would be preserved anyway and having routers make decisions based on an erroneous IPv6 header won't really hurt anybody (I still somehow find that hard to believe). > > What if we had a CRC-32 checksum in the IPv4 header, in > > the IPv6 header, in UDP Lite's header, in TCP's header, > > in the UDP header, well - just all over the place. > > The argument (far as I recall) was that things would be slower, not better. Sure, the checksum was removed to speed things up - but, see below... > > Then, we could just say "disable your checksums" to > > link layers when they notice a UDP Lite packet. It would > > even make sense to have a "corruption acceptable" bit > > somewhere in the IP header in such a scenario (I know, > > no bits to spare). > > > > Wouldn't that be better design? > > Define better. Again, the assumption is that not that many packets are Here's a try. Better = "guided by principles (*), not short-sighted performance optimizations (**)". (*) ... such as "IP over everything" (**) ... such as "let's speed things up" > corrupted, and that when they are their path doesn't matter so long as > the packets aren't accepted at the alleged destination. How would Assumption #1 is an ugly one because it breaks the "IP over everything" principle and now prevents us from cleanly realizing stuff like UDP Lite. Assumption #2 seems strange to me, but okay, if that was it, I could live with it (i.e. not call it ugly). > checksums everywhere help that? Is it necessary (and worth it) to do > more (the argument for IPv6 was "no")? Yeah, I'd say! :) Cheers, Michael From touch at ISI.EDU Wed Feb 16 02:01:34 2005 From: touch at ISI.EDU (Joe Touch) Date: Wed, 16 Feb 2005 02:01:34 -0800 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: References: Message-ID: <421319FE.10002@isi.edu> Vadim Antonov wrote: ... > A useful security cannot be implemented at IP or TCP layer, because > security is only as good as its weakest point. Any strong security > requires entity authentication (which means, pretty much, authenticating a > physical user, a server process, and determination of trustworthiness of > an execution environment), access authorization (meaning determination of > which entity is allowed to do what), privacy and integrity protection > (encryption and message authentication), intrusion detection and reaction > capabilities, and key managament. ... > Essentially, the only way to build a reasonably secure system, having an > elusive but quite important property called "compartmentalization" is to > do it at the application/middleware level, and treat networks as always > insecure. > > This also means that attempts to drag security down to the network layer > are, at best, misguided. Security at each layer is designed for a different purpose. The only way to secure the network layer information (used at the network layer (e.g., for forwarding) and transport - for connection demuxing) is to secure the network layer header. No amount of application layer protection works there, unless you terminate the app layer at each hop and forward there (e.g., P2P). Are you proposing that? Sure, you can require only security at the app layer, which means that layer is going to get a lot of junk misforwarded and mis-demuxed that it has to invest effort - at the routers and the endstations - processing. Security at other layers helps winnow that junk before it consumes that effort inappropriately. Security at any ONE layer is doomed. Security is a multilayer issue; always has been. We can always talk, in the context of multilayer security, at what layer to address a given vulnerability. Joe -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 254 bytes Desc: OpenPGP digital signature Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050216/59761adf/signature.bin From touch at ISI.EDU Wed Feb 16 02:29:47 2005 From: touch at ISI.EDU (Joe Touch) Date: Wed, 16 Feb 2005 02:29:47 -0800 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <1108546313.4764.160.camel@lap10-c703.uibk.ac.at> References: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> <42123399.7050508@isi.edu> <1108489580.4768.228.camel@lap10-c703.uibk.ac.at> <42124C81.1000902@isi.edu> <1108496523.4768.244.camel@lap10-c703.uibk.ac.at> <421283AE.6020508@isi.edu> <1108546313.4764.160.camel@lap10-c703.uibk.ac.at> Message-ID: <4213209B.70200@isi.edu> Michael Welzl wrote: >>>I also searched RFC 2460 for this once, but couldn't find anything. >>>Still, that doesn't make sense to me. What if you transfer an >>>IPv6 packet across a link layer that does *NOT* check your data - >>>why wouldn't this cause harm to the source / destination addresses, >>>and therefore cause wrong router behavior downstream? >> >>It doesn't cause harm at the destination, since there's supposed to be a >>transport checksum (even for ICMP - one was added). I suppose there's no >>perceived harm in sending some corrupted traffic through the wrong paths. >> >>Note - this is designed to address corruption, not malicious behavior. > > Okay, I agree that end-to-end integrity is preserved, nothing > wrong about that. It's just that I find it hard to believe > that there's no perceived harm in having routers make decisions > based upon an erroneous IP header. Is sending some corrupted > traffic through the wrong paths really all that can happen? > Hm, it could affect the hop limit, too, but who cares. > > Okay, so what about the "payload length" field - couldn't > it lead to errors (a router allocates a buffer that is way > too large ... hmmm, do we care?) ... I don't know if we care much more than we care about misuse of the outgoing link. > I'm unsure, but still, I find it hard to believe. Anyone on the list more familiar with the history of IPv6 care to comment? >>And >> >>>what about the other fields - say, "next header"; a checksum >>>error could turn a TCP packet into a UDP packet, change the >>>payload length field, whatever. >> >>Sure - but then that checksum should either match or somebody's >>overwriting packets anyway, which they can do even if there are IP >>checksums. Onec you overwrite things, all bets are off. > > I don't get that. There's no IPv6 checksum, so these fields > could be changed by corruption and noone would notice. We > could say that we don't care about the "next header" field > too, because that's relevant in the end system, where the > pseudo header will take effect. Again, that appears to be the logic. > So, I wonder: was the reason not to have an IPv6 checksum: > > * because the assumption was that link layers provide > strong CRC protection anyway ( = "IP no longer over everything") > > or > > * because end-to-end integrity would be preserved anyway > and having routers make decisions based on an erroneous > IPv6 header won't really hurt anybody (I still somehow > find that hard to believe). Combination of the two - that link layers that want to check are better than what IP provided, and that when they don't it's good enough to check at the transport layer. There's also a notion (AFAIR) that IP checksums don't catch malicious behavior or much about link corruption (which is more about bursts than random, typically) - they have tended to catch buggy implementations, and the effort of putting that check in all routers isn't worth using it as a "can you implement a router properly" check (which is more what it provided, i.e.). >>>What if we had a CRC-32 checksum in the IPv4 header, in >>>the IPv6 header, in UDP Lite's header, in TCP's header, >>>in the UDP header, well - just all over the place. >> >>The argument (far as I recall) was that things would be slower, not better. > > Sure, the checksum was removed to speed things up - but, > see below... > >>>Then, we could just say "disable your checksums" to >>>link layers when they notice a UDP Lite packet. It would >>>even make sense to have a "corruption acceptable" bit >>>somewhere in the IP header in such a scenario (I know, >>>no bits to spare). >>> >>>Wouldn't that be better design? >> >>Define better. Again, the assumption is that not that many packets are > > Here's a try. Better = "guided by principles (*), not short-sighted > performance optimizations (**)". > > (*) ... such as "IP over everything" > (**) ... such as "let's speed things up" How about "doesn't help for the purpose intended"? I.e., if the checksum basically helped find buggy router implementations, not avoiding link errors as much (typically, as will be discussed), then it's more effort than it's worth. The idea, again, AFAIR, is that buggy links ought to use link error detection, not rely on IP layer protection. This may indeed be counter to the UDP-Lite assumption, but I never bought that one per se anyway. All this work to get arbitrarily corrupted data to the app seems misguided, moreso (IMO) than forwarding buggy packets even. >>corrupted, and that when they are their path doesn't matter so long as >>the packets aren't accepted at the alleged destination. How would > > Assumption #1 is an ugly one because it breaks the "IP over > everything" principle and now prevents us from cleanly realizing > stuff like UDP Lite. Whoa Nelly! "cleanly realizing stuff like UDP Lite" - poster-child for unclean layer violation, in how it wants the _app_ to change how the transport interacts with the link layer? I need to clean my keyboard for just typing about it ;-) > Assumption #2 seems strange to me, but okay, if that was it, > I could live with it (i.e. not call it ugly). I think it's more like "not worth the effort" than "speeds things up". The checksum isn't a particularly slow part; it can force a store-and-forward delay, rather than cut-through processing, as well as an additional $0.01 of hardware (if that), but it's not slow. Joe -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 254 bytes Desc: OpenPGP digital signature Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050216/c787ef5e/signature-0001.bin From avg at kotovnik.com Wed Feb 16 04:01:12 2005 From: avg at kotovnik.com (Vadim Antonov) Date: Wed, 16 Feb 2005 04:01:12 -0800 (PST) Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: <421319FE.10002@isi.edu> Message-ID: On Wed, 16 Feb 2005, Joe Touch wrote: > Security at each layer is designed for a different purpose. > > The only way to secure the network layer information (used at the > network layer (e.g., for forwarding) and transport - for connection > demuxing) is to secure the network layer header. No amount of > application layer protection works there, unless you terminate the app > layer at each hop and forward there (e.g., P2P). Are you proposing that? I propose doing security at the appropriate layer. Data encryption is appropriate at the application-to-application layer, not at the network-to-network (or even host-to-host) layer. Of course, some paranoid people encrypt data at both network and application layers, but this does not increase security much. > Sure, you can require only security at the app layer, which means that > layer is going to get a lot of junk misforwarded and mis-demuxed that it > has to invest effort - at the routers and the endstations - processing. > Security at other layers helps winnow that junk before it consumes that > effort inappropriately. Misrouted or mis-demuxed junk means only that the routing equipment/software is broken. Working equipment doesn't do that, generally, and it never was a problem, if the routers or switches themselves aren't compromised (and when an end-host OS is compromised you've got problems way more serious than some additional junk in the network traffic). > Security at any ONE layer is doomed. Security is a multilayer issue; > always has been. We can always talk, in the context of multilayer > security, at what layer to address a given vulnerability. Of course, but what you need at the network layer is integrity and security of routing information and router OSes, not the integrity/security of user data. This is because data encryption in the network (not at end-points) leaves traffic vulnerable to the snooping or modification by network administrators, who (in most cases) aren't in the same group as service users, and, therefore, have different trustworthiness profile. On the other hand, they are in control of network routing, so protection of routing infrastructure is appropriately done at the network layer. What network layer can do for e2e security is to provide some support for resistance to flooding attacks. Unfortunately, encryption is exactly the wrong way to achieve that, because it takes more resources to handle an encrypted packet, and the whole point of flooding DoS attack is to exhaust resources of a target system. VPNs don't do anything to protect routing information, and don't do anything to handle flooding attacks. They mostly create an illusion of network security, distracting people from addressing the real security threats. --vadim From dpreed at reed.com Wed Feb 16 04:55:13 2005 From: dpreed at reed.com (David P. Reed) Date: Wed, 16 Feb 2005 07:55:13 -0500 Subject: [e2e] Security implications blurring the name/address distinction In-Reply-To: <4213209B.70200@isi.edu> References: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> <42123399.7050508@isi.edu> <1108489580.4768.228.camel@lap10-c703.uibk.ac.at> <42124C81.1000902@isi.edu> <1108496523.4768.244.camel@lap10-c703.uibk.ac.at> <421283AE.6020508@isi.edu> <1108546313.4764.160.camel@lap10-c703.uibk.ac.at> <4213209B.70200@isi.edu> Message-ID: <421342B1.8090908@reed.com> We've strayed far afield from the original question, prompted by my digression about the history of "not securing" TCP. And it is great to have this discussion - it would have been even better to have this discussion years ago! Both Vadim and Joe make good points, from particular points of view. Since they are both right, I think it's worth teasing out the structure of the problem here. (Lloyd - in this I'm not intending to be sanctimonious, just slightly pedantic in hopes of better clarity). There are two end-to-end security properties under discussion here: 1) end-to-end messages contain names and data that are certain to be correct, without depending on the transport. (integrity and ) 2) endpoints' workload should be protected as much as possible from hostile actors. (protection from denial of service) There is a third security property that is somewhat controversial - end-to-end messages should be readable only by authorized recipients (CALEA-not). Vadim points out that confusing network addresses with endpoint names is both unnecessary and opens up risks to the first property. The cost of his proposal is maintaining two sets of names, exposing only the address hints to the network. This also helps manage the third property (though traffic analysis can reveal something about the messages if the addresses are too richly endowed - encrypted onion routing minimizes the third risk). Joe focuses on the need for measures to minimize the denial of service risk. There are other possible measures that could manage this, once one realizes that this risk has little to do with end-do-end naming, and everything to do with "universal addressibility". The risk of each node being able to address every physical node in the network, which is mostly a good thing, leads to all kinds of denial of service possibliities. This really has little to do with the "end-to-end" protocol (whether TCP or UDP), though. A much better formulation of the denial of service risk that arises due to network addressing would focus entirely on the IP layer, and not on TCP at all. Solutions to the denial of service issues would then focus on the work factor involved in a malicious actor being able to impose work on other actors, the cost of detecting and disciplining the malicious actors, and the cost of isolating disruption. One would then focus on IP layer solutions that address these problems. Such IP layer solutions might involve filtering gateways that have time-varying tokens for entry, time-varying addresses (changing your phone number is a lot easier than changing your name when harassed), network layer packet backtrace capability, eliminating or guarding "packet amplifiers" like broadcast, etc. But by confusing the layers, we confused the responsibility for very different security properties, and a better layer separation would have been a good idea! I shouldn't need to say it here, but TCP is not the perfect embodiment of the "end-to-end" principle, but instead is full of compromises. It's far more end-to-end than was SNA or Tymnet or NCP, some of its predecessors. The sharing of the name field was indeed such a compromise. We did it in the name of "saving header overhead", and because the address/name debate was not fully understood by the community. I feel bad, though, that the "security" community seems to take the blurred distinctions as a given, blurring their own notion of what makes a secure network. As Vadim rightly points out, security should not be the province solely of the network administration, yet that is where the burden is put by most corporations, including most major endpoint application vendors (e.g. Microsoft gets sucked into discussions about "open ports" as if that puts the end-to-end virus or worm security risk into the realm of firewalls to fix). From touch at ISI.EDU Wed Feb 16 04:58:32 2005 From: touch at ISI.EDU (Joe Touch) Date: Wed, 16 Feb 2005 04:58:32 -0800 Subject: [e2e] purpose of pseudo header in TCP checksum In-Reply-To: References: Message-ID: <42134378.1000007@isi.edu> Vadim Antonov wrote: > On Wed, 16 Feb 2005, Joe Touch wrote: > > >>Security at each layer is designed for a different purpose. >> >>The only way to secure the network layer information (used at the >>network layer (e.g., for forwarding) and transport - for connection >>demuxing) is to secure the network layer header. No amount of >>application layer protection works there, unless you terminate the app >>layer at each hop and forward there (e.g., P2P). Are you proposing that? > > I propose doing security at the appropriate layer. Data encryption is > appropriate at the application-to-application layer, not at the > network-to-network (or even host-to-host) layer. Data encryption should be secured by the source. The IP packet that causes the ICMP error is the source, and IPsec is what should secure (authenticate) it. IPsec on that packet is the appropriate level of security; no application security can authenticate the IP header, even when it becomes payload for the ICMP. > Of course, some paranoid people encrypt data at both network and > application layers, but this does not increase security much. Apps don't secure the IP layer; they often don't know what the IP header will be when the packet is emitted (leaving, e.g., source IP address unbound at the socket). >>Sure, you can require only security at the app layer, which means that >>layer is going to get a lot of junk misforwarded and mis-demuxed that it >>has to invest effort - at the routers and the endstations - processing. >>Security at other layers helps winnow that junk before it consumes that >>effort inappropriately. > > Misrouted or mis-demuxed junk means only that the routing > equipment/software is broken. If the link layer generates problems, the routing system can still work fine and be forwarding a packet correctly that some would consider 'misrouted'. > Working equipment doesn't do that, > generally, and it never was a problem, if the routers or switches > themselves aren't compromised (and when an end-host OS is compromised > you've got problems way more serious than some additional junk in the > network traffic). Or, as above, the link has errors that are undetected (lack of link checks, or turning them off for UDP Lite). >>Security at any ONE layer is doomed. Security is a multilayer issue; >>always has been. We can always talk, in the context of multilayer >>security, at what layer to address a given vulnerability. > > Of course, but what you need at the network layer is integrity and > security of routing information and router OSes, not the > integrity/security of user data. Security of the IP header used for forwarding too. > This is because data encryption in the network (not at end-points) leaves > traffic vulnerable to the snooping or modification by network > administrators, who (in most cases) aren't in the same group as service > users, and, therefore, have different trustworthiness profile. On the > other hand, they are in control of network routing, so protection of > routing infrastructure is appropriately done at the network layer. > > What network layer can do for e2e security is to provide some support for > resistance to flooding attacks. Or attacks on network-level systems, e.g. ICMP. Or attacks on protocols that include IP header information at their level - as we've been discussing for TCP, e.g. > Unfortunately, encryption is exactly the > wrong way to achieve that, because it takes more resources to handle an > encrypted packet, and the whole point of flooding DoS attack is to exhaust > resources of a target system. One point of a flooding attack is to exhaust resources. Other points are to disrupt services by killing connections, e.g., or interfering with PMTUD. > VPNs don't do anything to protect routing information, and don't do > anything to handle flooding attacks. They mostly create an illusion of > network security, distracting people from addressing the real security > threats. > > --vadim You've already spoken of that position before, but it is not relevant to this discussion. Joe -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 254 bytes Desc: OpenPGP digital signature Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050216/53265d12/signature.bin From touch at ISI.EDU Wed Feb 16 05:04:16 2005 From: touch at ISI.EDU (Joe Touch) Date: Wed, 16 Feb 2005 05:04:16 -0800 Subject: [e2e] Security implications blurring the name/address distinction In-Reply-To: <421342B1.8090908@reed.com> References: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> <42123399.7050508@isi.edu> <1108489580.4768.228.camel@lap10-c703.uibk.ac.at> <42124C81.1000902@isi.edu> <1108496523.4768.244.camel@lap10-c703.uibk.ac.at> <421283AE.6020508@isi.edu> <1108546313.4764.160.camel@lap10-c703.uibk.ac.at> <4213209B.70200@isi.edu> <421342B1.8090908@reed.com> Message-ID: <421344D0.1060506@isi.edu> David P. Reed wrote: > We've strayed far afield from the original question, prompted by my > digression about the history of "not securing" TCP. And it is great to > have this discussion - it would have been even better to have this > discussion years ago! > > Both Vadim and Joe make good points, from particular points of view. > Since they are both right, I think it's worth teasing out the structure > of the problem here. (Lloyd - in this I'm not intending to be > sanctimonious, just slightly pedantic in hopes of better clarity). > > There are two end-to-end security properties under discussion here: > > 1) end-to-end messages contain names and data that are certain to be > correct, without depending on the transport. (integrity and ) > > 2) endpoints' workload should be protected as much as possible from > hostile actors. (protection from denial of service) > > There is a third security property that is somewhat controversial - > end-to-end messages should be readable only by authorized recipients > (CALEA-not). > > Vadim points out that confusing network addresses with endpoint names is > both unnecessary and opens up risks to the first property. The cost of > his proposal is maintaining two sets of names, exposing only the address > hints to the network. This also helps manage the third property (though > traffic analysis can reveal something about the messages if the > addresses are too richly endowed - encrypted onion routing minimizes the > third risk). Having separate network addresses still requires that they be secured somehow, or else spoofing can still occur. Granted, that does help decouple the TCP vs. IP solution space for such solutions, though. > Joe focuses on the need for measures to minimize the denial of service > risk. There are other possible measures that could manage this, once > one realizes that this risk has little to do with end-do-end naming, and > everything to do with "universal addressibility". Actually, I'm dealing with all sorts of attacks, not just DOS. I don't consider PMTU distruption a DOS attack, nor do I consider TCP RSTs one either. These are attacks on connections, not on bandwidth or buffer space, which is what I think of as a defining property of a DOS attack. > The risk of each node being able to address every physical node in the > network, which is mostly a good thing, leads to all kinds of denial of > service possibliities. > > This really has little to do with the "end-to-end" protocol (whether TCP > or UDP), though. > > A much better formulation of the denial of service risk that arises due > to network addressing would focus entirely on the IP layer, and not on > TCP at all. As above, these aren't DOS attacks. They are attacks on the protocol. It's natural to assume that attacks on a specific protocol are solved by securing that protocol - in this case TCP. However, because TCP uses the pseudoheader, and because that information isn't authenticated, TCP is expected to take up the slack, and with mechanisms that are correlation-based, not security-based. > Solutions to the denial of service issues would then focus on the work > factor involved in a malicious actor being able to impose work on other > actors, the cost of detecting and disciplining the malicious actors, and > the cost of isolating disruption. One would then focus on IP layer > solutions that address these problems. Such IP layer solutions might > involve filtering gateways that have time-varying tokens for entry, > time-varying addresses (changing your phone number is a lot easier than > changing your name when harassed), network layer packet backtrace > capability, eliminating or guarding "packet amplifiers" like broadcast, > etc. > > But by confusing the layers, we confused the responsibility for very > different security properties, and a better layer separation would have > been a good idea! > > I shouldn't need to say it here, but TCP is not the perfect embodiment > of the "end-to-end" principle, but instead is full of compromises. It's > far more end-to-end than was SNA or Tymnet or NCP, some of its > predecessors. The sharing of the name field was indeed such a > compromise. We did it in the name of "saving header overhead", and > because the address/name debate was not fully understood by the community. > > I feel bad, though, that the "security" community seems to take the > blurred distinctions as a given, blurring their own notion of what makes > a secure network. As Vadim rightly points out, security should not be > the province solely of the network administration, yet that is where the > burden is put by most corporations, including most major endpoint > application vendors (e.g. Microsoft gets sucked into discussions about > "open ports" as if that puts the end-to-end virus or worm security risk > into the realm of firewalls to fix). Security of ICMP should definitely be the purvue of a network administrator. Applications can't protect connections that are cut from under their feet. Joe -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 254 bytes Desc: OpenPGP digital signature Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050216/082411c8/signature.bin From dpreed at reed.com Wed Feb 16 06:43:57 2005 From: dpreed at reed.com (David P. Reed) Date: Wed, 16 Feb 2005 09:43:57 -0500 Subject: [e2e] Security implications blurring the name/address distinction In-Reply-To: <421344D0.1060506@isi.edu> References: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> <42123399.7050508@isi.edu> <1108489580.4768.228.camel@lap10-c703.uibk.ac.at> <42124C81.1000902@isi.edu> <1108496523.4768.244.camel@lap10-c703.uibk.ac.at> <421283AE.6020508@isi.edu> <1108546313.4764.160.camel@lap10-c703.uibk.ac.at> <4213209B.70200@isi.edu> <421342B1.8090908@reed.com> <421344D0.1060506@isi.edu> Message-ID: <42135C2D.5050602@reed.com> Joe - the classic security community definition of denial of service includes TCP RST and PMTU attacks. Neither exposes the content of a message or corrupts the content of message - they merely interfere with reliability of delivery. You may want to subdivide "denial of service" into subcategories. But ultimately, those attacks deny service. And that's my point. There is nothing in the classification of security risks that says "denial of service" is protocol-independent. From touch at ISI.EDU Wed Feb 16 08:26:09 2005 From: touch at ISI.EDU (Joe Touch) Date: Wed, 16 Feb 2005 08:26:09 -0800 Subject: [e2e] Security implications blurring the name/address distinction In-Reply-To: <42135C2D.5050602@reed.com> References: <20050215143559.9BE7D872CF@mercury.lcs.mit.edu> <1108483506.4768.206.camel@lap10-c703.uibk.ac.at> <42123399.7050508@isi.edu> <1108489580.4768.228.camel@lap10-c703.uibk.ac.at> <42124C81.1000902@isi.edu> <1108496523.4768.244.camel@lap10-c703.uibk.ac.at> <421283AE.6020508@isi.edu> <1108546313.4764.160.camel@lap10-c703.uibk.ac.at> <4213209B.70200@isi.edu> <421342B1.8090908@reed.com> <421344D0.1060506@isi.edu> <42135C2D.5050602@reed.com> Message-ID: <42137421.2060803@isi.edu> David P. Reed wrote: > Joe - > > the classic security community definition of denial of service includes > TCP RST and PMTU attacks. Neither exposes the content of a message or > corrupts the content of message - they merely interfere with reliability > of delivery. > > You may want to subdivide "denial of service" into subcategories. But > ultimately, those attacks deny service. And that's my point. > > There is nothing in the classification of security risks that says > "denial of service" is protocol-independent. There's a big difference between attacks that overwhelm resources and ones that kill connections or drop packets. In my reading of the security community work, DOS tends to focus on the resource-starvation - by adding new connections, by overwhelming forrwarding or security processing, by consuming buffers. The service is denied because of OTHER THINGS going on. They're indirect attacks on particular connections. TCP RST and PMTU are direct attacks. Joe -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 254 bytes Desc: OpenPGP digital signature Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050216/92e359cc/signature-0001.bin From djw1005 at cam.ac.uk Wed Feb 16 17:31:08 2005 From: djw1005 at cam.ac.uk (Damon Wischik) Date: Thu, 17 Feb 2005 01:31:08 +0000 (GMT) Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: <000a01c50f95$98bfd660$f01aeb80@snow> References: <000a01c50f95$98bfd660$f01aeb80@snow> Message-ID: Roy Xu wrote: > I'm looking for a pointer to literatures that link the > TCP's (discrete) AIMD to Kelly's (continuous) control formulation. Kelly's continuous-time formulation uses a differential equation model (also called a fluid model) for TCP. You should look at the literature which describes this fluid model, starting with "A Fluid-based Analysis of a Network of AQM Routers Supporting TCP Flows with an Application to RED", V. Misra, W. Gong, D. Towsley, SIGCOMM 2000. There are many links provided at http://gaia.cs.umass.edu/fluid/ I have collected some further links at http://www.cs.ucl.ac.uk/staff/D.Wischik/Interests/Topics/tcpqueue.html Look especially at these papers: "Using partial differential equations to model TCP mice and elephants in large IP networks." Marco Ajmone Marsan, Michele Gatetto, Paolo Giaccone, Emilio Leonardi, Enrico Schiattarella, Alessandro Tarello. "A mean-field model for multiple TCP connections through a buffer implementing RED", Francois Baccelli, David R. McDonald, Julien Reynier. Performance Evaluation, 2002; Damon. From faber at ISI.EDU Thu Feb 17 11:20:35 2005 From: faber at ISI.EDU (Ted Faber) Date: Thu, 17 Feb 2005 11:20:35 -0800 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: References: <000a01c50f95$98bfd660$f01aeb80@snow> Message-ID: <20050217192035.GA77148@pun.isi.edu> On Thu, Feb 17, 2005 at 01:31:08AM +0000, Damon Wischik wrote: > > Roy Xu wrote: > >I'm looking for a pointer to literatures that link the > >TCP's (discrete) AIMD to Kelly's (continuous) control formulation. > > Kelly's continuous-time formulation uses a differential equation model > (also called a fluid model) for TCP. You should look at the literature > which describes this fluid model, starting with > > "A Fluid-based Analysis of a Network of AQM Routers Supporting TCP Flows > with an Application to RED", V. Misra, W. Gong, D. Towsley, SIGCOMM 2000. Fluid flow congestion control analysis goes beck to these guys at least: %A D. Mitra %A T. Seery %T Dynamic Adaptive Windows for High Speed Data Networks: Theory and %Simulation %J Proc. SIGCOMM Symposium on Communications Architectures and Protocols %P 30-29 %I ACM SIGCOMM %C Philadelphia, PA %D Sept 24-27, 1990 -- Ted Faber http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.asc Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050217/af6b93b3/attachment.bin From cannara at attglobal.net Thu Feb 17 21:12:58 2005 From: cannara at attglobal.net (Cannara) Date: Thu, 17 Feb 2005 21:12:58 -0800 Subject: [e2e] link between Kelly's control and TCP's AIMD References: <000a01c50f95$98bfd660$f01aeb80@snow> <20050217192035.GA77148@pun.isi.edu> Message-ID: <4215795A.CC886DFD@attglobal.net> And, without understanding/modelling TCP's nonlinear behaviors, it will only serve for gross predictions. Alex Ted Faber wrote: > > On Thu, Feb 17, 2005 at 01:31:08AM +0000, Damon Wischik wrote: > > > > Roy Xu wrote: > > >I'm looking for a pointer to literatures that link the > > >TCP's (discrete) AIMD to Kelly's (continuous) control formulation. > > > > Kelly's continuous-time formulation uses a differential equation model > > (also called a fluid model) for TCP. You should look at the literature > > which describes this fluid model, starting with > > > > "A Fluid-based Analysis of a Network of AQM Routers Supporting TCP Flows > > with an Application to RED", V. Misra, W. Gong, D. Towsley, SIGCOMM 2000. > > Fluid flow congestion control analysis goes beck to these guys at least: > > %A D. Mitra > %A T. Seery > %T Dynamic Adaptive Windows for High Speed Data Networks: Theory and > %Simulation > %J Proc. SIGCOMM Symposium on Communications Architectures and Protocols > %P 30-29 > %I ACM SIGCOMM > %C Philadelphia, PA > %D Sept 24-27, 1990 > > -- > Ted Faber > http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.asc > Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG From Jon.Crowcroft at cl.cam.ac.uk Fri Feb 18 00:02:35 2005 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Fri, 18 Feb 2005 08:02:35 +0000 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: Message from Ted Faber of "Thu, 17 Feb 2005 11:20:35 PST." <20050217192035.GA77148@pun.isi.edu> Message-ID: so the idea of packet conservation and ack clocking was completely there in Van's original 88 paper. the continuous control theoretic model was also pretty much there - and is partly from the early DEC work but i would guess that if you look at Davies work and some of the early french packet switched net work, there was some analogy with fluids already in the back of peoples minds. I guess the interesting thing is how much work is based on something which for a long time was pretty poor approx (the percentage of flows that make it into a steady state being so low for ages, that the notion that each new packet is only sent in because an ack came out was kind of fantasy land) - when Van did this mid to late 80s, the net was full of long usenet and ftp transfers - by early 90s, it was full of tiny intsy web transfers - nowadays, i guess we may be nearly back in the regime where there's a reasonable percentage of longish (p2p) tcp transfers although the ratio of access link speed to core is so huge, that from the core perspective, most tcp flows are pretty tiny things - its only really at the inter-pop traffic that the aggregate flow might look like something fluid approximations work for...but then it aint a TCP AIMD process, its an aggregate of a set of flows starting and stopping which is quite a different thing... as steven hand just said to me - reasoning by analogy is like trying to fix a car with a leaky screwdriver In missive <20050217192035.GA77148 at pun.isi.edu>, Ted Faber typed: >> >>--ReaqsoxgOBHFXBhH >>Content-Type: text/plain; charset=us-ascii >>Content-Disposition: inline >> >>On Thu, Feb 17, 2005 at 01:31:08AM +0000, Damon Wischik wrote: >>> >>> Roy Xu wrote: >>> >I'm looking for a pointer to literatures that link the >>> >TCP's (discrete) AIMD to Kelly's (continuous) control formulation. >>> >>> Kelly's continuous-time formulation uses a differential equation model >>> (also called a fluid model) for TCP. You should look at the literature >>> which describes this fluid model, starting with >>> >>> "A Fluid-based Analysis of a Network of AQM Routers Supporting TCP Flows >>> with an Application to RED", V. Misra, W. Gong, D. Towsley, SIGCOMM 2000. >> >>Fluid flow congestion control analysis goes beck to these guys at least: >> >>%A D. Mitra >>%A T. Seery >>%T Dynamic Adaptive Windows for High Speed Data Networks: Theory and >>%Simulation >>%J Proc. SIGCOMM Symposium on Communications Architectures and Protocols >>%P 30-29 >>%I ACM SIGCOMM >>%C Philadelphia, PA >>%D Sept 24-27, 1990 >> >> >>-- >>Ted Faber >>http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.asc >>Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG >> >>--ReaqsoxgOBHFXBhH >>Content-Type: application/pgp-signature >>Content-Disposition: inline >> >>-----BEGIN PGP SIGNATURE----- >>Version: GnuPG v1.4.0 (FreeBSD) >> >>iD8DBQFCFO6DaUz3f+Zf+XsRAn3BAKDWWh+sku9pRJawX4ipToWrm7s7UwCgjqs7 >>TiD1WS+S5ILKts4qO+fFmoc= >>=d0Jg >>-----END PGP SIGNATURE----- >> >>--ReaqsoxgOBHFXBhH-- cheers jon From puddinghead_wilson007 at yahoo.co.uk Fri Feb 18 05:01:59 2005 From: puddinghead_wilson007 at yahoo.co.uk (Puddinhead Wilson) Date: Fri, 18 Feb 2005 13:01:59 +0000 (GMT) Subject: [e2e] a problem... In-Reply-To: <20050114235829.31743.qmail@web25703.mail.ukl.yahoo.com> Message-ID: <20050218130159.46635.qmail@web25708.mail.ukl.yahoo.com> So while discussing this with folks I ran into another interesting case We have N1------------------>N2 | | -------------------> one wishes to have redundant links, so that if N1 fails, N2 should be used the links are long and huge.. Some folks prefer to send data over both links simultaneously and let the receiver side decide which one to use...funnily they still need to "signal" the failure of a link across N1 and N2, rather than simply rely on absence of clock (??)..the going word for it is APS. Another option could be to let N1 decide which link to use. Obviously both links have to terminate on different ports, so it wont matter one way or the other... the clock detection still seems simpler :(, another advantage being you could use "both", but lets leave that aside for now.. also say: if we do "short 2 wires" and send data over both, we need an arbitrator/switch to decide which wire to use. should this arbitrator be on the ingress side or the egress side? --- Puddinhead Wilson wrote: > exactly, > > A constant DC is "no information" :) > > You best put my question. > > but it's "presence or absence" is. > > Say for example I have > N1--->N2 > if both N1 and N2 use the same frequency/channel etc > to txmt, it would result in distortion of > information > > What is that component that I always need to > transmit > the information but does use up my channel capacity. > ___________________________________________________________ ALL-NEW Yahoo! Messenger - all new features - even more fun! http://uk.messenger.yahoo.com From faber at ISI.EDU Fri Feb 18 10:18:52 2005 From: faber at ISI.EDU (Ted Faber) Date: Fri, 18 Feb 2005 10:18:52 -0800 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: <4215795A.CC886DFD@attglobal.net> References: <000a01c50f95$98bfd660$f01aeb80@snow> <20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> Message-ID: <20050218181852.GO90698@pun.isi.edu> On Thu, Feb 17, 2005 at 09:12:58PM -0800, Cannara wrote: > And, without understanding/modelling TCP's nonlinear behaviors, it will only > serve for gross predictions. Without defending Mitra and Seery's work, which stands perfectly well on its own, the point was that fluid flow congestion analysis has a long and often ignored history. -- Ted Faber http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.asc Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050218/cde9e635/attachment.bin From faber at ISI.EDU Fri Feb 18 10:33:43 2005 From: faber at ISI.EDU (Ted Faber) Date: Fri, 18 Feb 2005 10:33:43 -0800 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: References: <20050217192035.GA77148@pun.isi.edu> Message-ID: <20050218183343.GP90698@pun.isi.edu> On Fri, Feb 18, 2005 at 08:02:35AM +0000, Jon Crowcroft wrote: > so the idea of packet conservation and ack clocking was completely > there in Van's original 88 paper. the continuous control theoretic > model was also pretty much there - and is partly from the early DEC > work but i would guess that if you look at Davies work and some of the > early french packet switched net work, there was some analogy with > fluids already in the back of peoples minds. I said "at least." I'm not asserting that they were the first, just that the ideas and analysis have been around for quite some time. I wouldn't claim to be definitive without digging back around in my congestion library. > > I guess the interesting thing is how much work is based on something > which for a long time was pretty poor approx (the percentage of flows > that make it into a steady state being so low for ages, that the > notion that each new packet is only sent in because an ack came out > was kind of fantasy land) - when Van did this mid to late 80s, the net > was full of long usenet and ftp transfers - by early 90s, it was full > of tiny intsy web transfers - nowadays, i guess we may be nearly back > in the regime where there's a reasonable percentage of longish (p2p) > tcp transfers although the ratio of access link speed to core is so > huge, that from the core perspective, most tcp flows are pretty tiny > things - its only really at the inter-pop traffic that the aggregate > flow might look like something fluid approximations work for...but > then it aint a TCP AIMD process, its an aggregate of a set of flows > starting and stopping which is quite a different thing... Those darn transients and corner cases sure make thing interesting when the're neither so transient nor so uncommon, that's for sure. If you're arguing that there's still a lot to do in congestion control, I certainly agree. :-) (And I share your surprise that such a gross approximation still yielded significant improvements.) > as steven hand just said to me - reasoning by analogy is like trying > to fix a car with a leaky screwdriver And yet without some analogies to guide us, people don't do very well in exploring the unknown. -- Ted Faber http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.asc Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050218/62b91c7d/attachment.bin From mascolo at poliba.it Fri Feb 18 12:11:31 2005 From: mascolo at poliba.it (Saverio Mascolo) Date: Fri, 18 Feb 2005 21:11:31 +0100 Subject: [e2e] link between Kelly's control and TCP's AIMD References: <000a01c50f95$98bfd660$f01aeb80@snow><20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> Message-ID: <003501c515f6$0989efa0$723bccc1@poliba.it> why do you think TCP congestion control is not linear? I tend to agree with Van Jacobson when syas that a network is to a large extend a linear system made of integrators, delays and gains. Saverio ----- Original Message ----- From: "Cannara" To: Sent: Friday, February 18, 2005 6:12 AM Subject: Re: [e2e] link between Kelly's control and TCP's AIMD > And, without understanding/modelling TCP's nonlinear behaviors, it will only > serve for gross predictions. > > Alex > > Ted Faber wrote: > > > > On Thu, Feb 17, 2005 at 01:31:08AM +0000, Damon Wischik wrote: > > > > > > Roy Xu wrote: > > > >I'm looking for a pointer to literatures that link the > > > >TCP's (discrete) AIMD to Kelly's (continuous) control formulation. > > > > > > Kelly's continuous-time formulation uses a differential equation model > > > (also called a fluid model) for TCP. You should look at the literature > > > which describes this fluid model, starting with > > > > > > "A Fluid-based Analysis of a Network of AQM Routers Supporting TCP Flows > > > with an Application to RED", V. Misra, W. Gong, D. Towsley, SIGCOMM 2000. > > > > Fluid flow congestion control analysis goes beck to these guys at least: > > > > %A D. Mitra > > %A T. Seery > > %T Dynamic Adaptive Windows for High Speed Data Networks: Theory and > > %Simulation > > %J Proc. SIGCOMM Symposium on Communications Architectures and Protocols > > %P 30-29 > > %I ACM SIGCOMM > > %C Philadelphia, PA > > %D Sept 24-27, 1990 > > > > -- > > Ted Faber > > http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.asc > > Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG > From cannara at attglobal.net Sat Feb 19 00:00:23 2005 From: cannara at attglobal.net (Cannara) Date: Sat, 19 Feb 2005 00:00:23 -0800 Subject: [e2e] link between Kelly's control and TCP's AIMD References: <000a01c50f95$98bfd660$f01aeb80@snow> <20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> <20050218181852.GO90698@pun.isi.edu> Message-ID: <4216F217.647A24AC@attglobal.net> I don't know "ignored" -- some friends (NetPredict) have a patent on just such statistical analyses applied to TCP performance products. It's the nonlinearity that makes any linear differential modelling grossly approximate. We have to remember some fundamentals -- at least that fluids are variously compressible & reorderable, unlike packets, and loss of molecules is not an option. :] Alex Ted Faber wrote: > > On Thu, Feb 17, 2005 at 09:12:58PM -0800, Cannara wrote: > > And, without understanding/modelling TCP's nonlinear behaviors, it will only > > serve for gross predictions. > > Without defending Mitra and Seery's work, which stands perfectly well on > its own, the point was that fluid flow congestion analysis has a long > and often ignored history. > > -- > Ted Faber > http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.asc > Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG > From cannara at attglobal.net Sat Feb 19 00:12:52 2005 From: cannara at attglobal.net (Cannara) Date: Sat, 19 Feb 2005 00:12:52 -0800 Subject: [e2e] a problem... References: <20050218130159.46635.qmail@web25708.mail.ukl.yahoo.com> Message-ID: <4216F504.37BAE29@attglobal.net> Why not go to the folks who make large switch fabric chips, with redundant, failover serial links, and see what they do. Maybe Vitesse (Gigastream...), etc.? Clock presence doesn't mean data presence, by the way. And, DC polarity is information -- pay phones have always returned your coins by reversal from the CO. Alex Puddinhead Wilson wrote: > > So while discussing this with folks I ran into another > interesting case > > We have > N1------------------>N2 > | | > -------------------> > > one wishes to have redundant links, so that if N1 > fails, N2 should be used the links are long and huge.. > > Some folks prefer to send data over both links > simultaneously and let the receiver side decide which > one to use...funnily they still need to "signal" the > failure of a link across N1 and N2, rather than simply > rely on absence of clock (??)..the going word for it > is APS. > > Another option could be to let N1 decide which link to > use. > > Obviously both links have to terminate on different > ports, so it wont matter one way or the other... the > clock detection still seems simpler :(, another > advantage being you could use "both", but lets leave > that aside for now.. > > also say: > if we do "short 2 wires" and send data over both, > we need an arbitrator/switch to decide which wire to > use. > should this arbitrator be on the ingress side or the > egress side? > > --- Puddinhead Wilson > wrote: > > exactly, > > > > A constant DC is "no information" :) > > > > You best put my question. > > > > but it's "presence or absence" is. > > > > Say for example I have > > N1--->N2 > > if both N1 and N2 use the same frequency/channel etc > > to txmt, it would result in distortion of > > information > > > > What is that component that I always need to > > transmit > > the information but does use up my channel capacity. From cannara at attglobal.net Sat Feb 19 00:18:42 2005 From: cannara at attglobal.net (Cannara) Date: Sat, 19 Feb 2005 00:18:42 -0800 Subject: [e2e] link between Kelly's control and TCP's AIMD References: <000a01c50f95$98bfd660$f01aeb80@snow><20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> <003501c515f6$0989efa0$723bccc1@poliba.it> Message-ID: <4216F662.2A59439D@attglobal.net> Well, only 15 years of modest network consulting for over 1000 modest companies from GM on down. :] Have VJ estimate how much delay is incurred by any current TCP under 1% random packet loss; then 0.1%. Or, have him or anyone else, itemize the differences among TCPs deployed by, say, the top 10 systems vendors. There've been many archived discussions of how TCP does & doesn't work as well as it should, given all the years that have passed, in which continued protocol development could have, but did not, occur. Thus the Internet -- a study in insecurity, capacity co-option, performance mediocrity, engineering bureaucracy... But, that's not what folks want to hear. Which is why the mediocrity continues and those of us in consulting & security have good jobs & good profits. :] Alex Saverio Mascolo wrote: > > why do you think TCP congestion control is not linear? I tend to agree with > Van Jacobson when syas that a network is to a large extend a linear system > made of integrators, delays and gains. > > Saverio > > ----- Original Message ----- > From: "Cannara" > To: > Sent: Friday, February 18, 2005 6:12 AM > Subject: Re: [e2e] link between Kelly's control and TCP's AIMD > > > And, without understanding/modelling TCP's nonlinear behaviors, it will > only > > serve for gross predictions. > > > > Alex > > > > Ted Faber wrote: > > > > > > On Thu, Feb 17, 2005 at 01:31:08AM +0000, Damon Wischik wrote: > > > > > > > > Roy Xu wrote: > > > > >I'm looking for a pointer to literatures that link the > > > > >TCP's (discrete) AIMD to Kelly's (continuous) control formulation. > > > > > > > > Kelly's continuous-time formulation uses a differential equation model > > > > (also called a fluid model) for TCP. You should look at the literature > > > > which describes this fluid model, starting with > > > > > > > > "A Fluid-based Analysis of a Network of AQM Routers Supporting TCP > Flows > > > > with an Application to RED", V. Misra, W. Gong, D. Towsley, SIGCOMM > 2000. > > > > > > Fluid flow congestion control analysis goes beck to these guys at least: > > > > > > %A D. Mitra > > > %A T. Seery > > > %T Dynamic Adaptive Windows for High Speed Data Networks: Theory and > > > %Simulation > > > %J Proc. SIGCOMM Symposium on Communications Architectures and Protocols > > > %P 30-29 > > > %I ACM SIGCOMM > > > %C Philadelphia, PA > > > %D Sept 24-27, 1990 > > > From mascolo at poliba.it Sat Feb 19 03:52:28 2005 From: mascolo at poliba.it (Saverio Mascolo) Date: Sat, 19 Feb 2005 12:52:28 +0100 Subject: [e2e] link between Kelly's control and TCP's AIMD References: <000a01c50f95$98bfd660$f01aeb80@snow><20050217192035.GA77148@pun.isi.edu><4215795A.CC886DFD@attglobal.net><003501c515f6$0989efa0$723bccc1@poliba.it> <4216F662.2A59439D@attglobal.net> Message-ID: <002b01c51679$7d0412c0$723bccc1@poliba.it> well this is not a technical answer! If you build a system made of linear subsystems (i.e. gains, delays and integrators) how can you get a nonlinear system? Saverio > Well, only 15 years of modest network consulting for over 1000 modest > companies from GM on down. :] > > Have VJ estimate how much delay is incurred by any current TCP under 1% random > packet loss; then 0.1%. Or, have him or anyone else, itemize the differences > among TCPs deployed by, say, the top 10 systems vendors. > > There've been many archived discussions of how TCP does & doesn't work as well > as it should, given all the years that have passed, in which continued > protocol development could have, but did not, occur. Thus the Internet -- a > study in insecurity, capacity co-option, performance mediocrity, engineering > bureaucracy... But, that's not what folks want to hear. Which is why the > mediocrity continues and those of us in consulting & security have good jobs & > good profits. :] > > Alex > > Saverio Mascolo wrote: > > > > why do you think TCP congestion control is not linear? I tend to agree with > > Van Jacobson when syas that a network is to a large extend a linear system > > made of integrators, delays and gains. > > > > Saverio > > > > ----- Original Message ----- > > From: "Cannara" > > To: > > Sent: Friday, February 18, 2005 6:12 AM > > Subject: Re: [e2e] link between Kelly's control and TCP's AIMD > > > > > And, without understanding/modelling TCP's nonlinear behaviors, it will > > only > > > serve for gross predictions. > > > > > > Alex > > > > > > Ted Faber wrote: > > > > > > > > On Thu, Feb 17, 2005 at 01:31:08AM +0000, Damon Wischik wrote: > > > > > > > > > > Roy Xu wrote: > > > > > >I'm looking for a pointer to literatures that link the > > > > > >TCP's (discrete) AIMD to Kelly's (continuous) control formulation. > > > > > > > > > > Kelly's continuous-time formulation uses a differential equation model > > > > > (also called a fluid model) for TCP. You should look at the literature > > > > > which describes this fluid model, starting with > > > > > > > > > > "A Fluid-based Analysis of a Network of AQM Routers Supporting TCP > > Flows > > > > > with an Application to RED", V. Misra, W. Gong, D. Towsley, SIGCOMM > > 2000. > > > > > > > > Fluid flow congestion control analysis goes beck to these guys at least: > > > > > > > > %A D. Mitra > > > > %A T. Seery > > > > %T Dynamic Adaptive Windows for High Speed Data Networks: Theory and > > > > %Simulation > > > > %J Proc. SIGCOMM Symposium on Communications Architectures and Protocols > > > > %P 30-29 > > > > %I ACM SIGCOMM > > > > %C Philadelphia, PA > > > > %D Sept 24-27, 1990 > > > > > From misra at cs.columbia.edu Sat Feb 19 05:17:59 2005 From: misra at cs.columbia.edu (Vishal Misra) Date: Sat, 19 Feb 2005 08:17:59 -0500 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: <4216F217.647A24AC@attglobal.net> References: <000a01c50f95$98bfd660$f01aeb80@snow> <20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> <20050218181852.GO90698@pun.isi.edu> <4216F217.647A24AC@attglobal.net> Message-ID: <42173C87.4020008@cs.columbia.edu> Cannara wrote: > I don't know "ignored" -- some friends (NetPredict) have a patent on just such > statistical analyses applied to TCP performance products. It's the > nonlinearity that makes any linear differential modelling grossly approximate. > > We have to remember some fundamentals -- at least that fluids are variously > compressible & reorderable, unlike packets, and loss of molecules is not an > option. :] > "fluid" modeling doesn't imply an application of Hydraulics or Bernoulloi's laws for protocol analysis. The "fluid" simply implies that the quantity that is being analyzed is continuous, and not discrete. Fluid models of TCP themselves come in two flavors - one that analyze some mean value, and hence are intrinsically continuous in nature (e.g., the Sigcomm 2000 paper mentioned earlier in the thread), and, - the other kind where the "fluid limit" or continuous property is obtained from a scaling of some parameters to infinity, e.g., the number of flows (the Performance 2002 paper mentioned in the same posting). The differential equations that are obtained are non-linear, simply being "fluid" does not constrain them to linearity. Additionally, stochastic differential equation based "fluid" models of TCP allow jump discontinuities. -Vishal -- http://www.cs.columbia.edu/~misra/ From cannara at attglobal.net Sat Feb 19 12:12:36 2005 From: cannara at attglobal.net (Alex Cannara) Date: Sat, 19 Feb 2005 12:12:36 -0800 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: <002b01c51679$7d0412c0$723bccc1@poliba.it> References: <000a01c50f95$98bfd660$f01aeb80@snow><20050217192035.GA77148@pun.isi.edu><4215795A.CC886DFD@attglobal.net><003501c515f6$0989efa0$723bccc1@poliba.it> <4216F662.2A59439D@attglobal.net> <002b01c51679$7d0412c0$723bccc1@poliba.it> Message-ID: <42179DB4.7090505@attglobal.net> Lordy, Lordy, since when is a threshold or timeout linear? If I drop 1% of a true fluid flow randomly, do I get 30% less out the other end of the pipe? One key to TCP's nonlinearity is its naive attempt to protect the network layer from congestion. The folks in the '80s who trembled with anger at Metcalfe's prediction of Internet collapse lived through some bad, near-miss scares, but instead of fixing IP and the network-layer systems, chose to make the transport layer the guardian of the net. This has always been a band aid, or kludge, depending on your upbringing, but it led to complacency and even religious fervor about "TCP", with cults even espousing foolery like "TCP Friendliness" and forclosing good ideas on doing a real fix for over a decade. Of course the Internet bureaucracy did deal with idiotic IPv4 addressing, over as many years, and did so well we have IPv6 (or we hope we never will have it)! {:o] Alex Saverio Mascolo wrote: > well this is not a technical answer! If you build a system made of linear > subsystems (i.e. gains, delays and integrators) how can you get a nonlinear > system? > > Saverio > > >>Well, only 15 years of modest network consulting for over 1000 modest >>companies from GM on down. :] >> >>Have VJ estimate how much delay is incurred by any current TCP under 1% > > random > >>packet loss; then 0.1%. Or, have him or anyone else, itemize the > > differences > >>among TCPs deployed by, say, the top 10 systems vendors. >> >>There've been many archived discussions of how TCP does & doesn't work as > > well > >>as it should, given all the years that have passed, in which continued >>protocol development could have, but did not, occur. Thus the Internet -- > > a > >>study in insecurity, capacity co-option, performance mediocrity, > > engineering > >>bureaucracy... But, that's not what folks want to hear. Which is why the >>mediocrity continues and those of us in consulting & security have good > > jobs & > >>good profits. :] >> >>Alex >> >>Saverio Mascolo wrote: >> >>>why do you think TCP congestion control is not linear? I tend to agree > > with > >>>Van Jacobson when syas that a network is to a large extend a linear > > system > >>>made of integrators, delays and gains. >>> >>>Saverio >>> >>>----- Original Message ----- >>>From: "Cannara" >>>To: >>>Sent: Friday, February 18, 2005 6:12 AM >>>Subject: Re: [e2e] link between Kelly's control and TCP's AIMD >>> >>> >>>>And, without understanding/modelling TCP's nonlinear behaviors, it > > will > >>>only >>> >>>>serve for gross predictions. >>>> >>>>Alex >>>> >>>>Ted Faber wrote: >>>> >>>>>On Thu, Feb 17, 2005 at 01:31:08AM +0000, Damon Wischik wrote: >>>>> >>>>>>Roy Xu wrote: >>>>>> >>>>>>>I'm looking for a pointer to literatures that link the >>>>>>>TCP's (discrete) AIMD to Kelly's (continuous) control > > formulation. > >>>>>>Kelly's continuous-time formulation uses a differential equation > > model > >>>>>>(also called a fluid model) for TCP. You should look at the > > literature > >>>>>>which describes this fluid model, starting with >>>>>> >>>>>>"A Fluid-based Analysis of a Network of AQM Routers Supporting TCP >>> >>>Flows >>> >>>>>>with an Application to RED", V. Misra, W. Gong, D. Towsley, > > SIGCOMM > >>>2000. >>> >>>>>Fluid flow congestion control analysis goes beck to these guys at > > least: > >>>>>%A D. Mitra >>>>>%A T. Seery >>>>>%T Dynamic Adaptive Windows for High Speed Data Networks: Theory and >>>>>%Simulation >>>>>%J Proc. SIGCOMM Symposium on Communications Architectures and > > Protocols > >>>>>%P 30-29 >>>>>%I ACM SIGCOMM >>>>>%C Philadelphia, PA >>>>>%D Sept 24-27, 1990 >>>>> From cannara at attglobal.net Sat Feb 19 12:24:42 2005 From: cannara at attglobal.net (Alex Cannara) Date: Sat, 19 Feb 2005 12:24:42 -0800 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: <42173C87.4020008@cs.columbia.edu> References: <000a01c50f95$98bfd660$f01aeb80@snow> <20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> <20050218181852.GO90698@pun.isi.edu> <4216F217.647A24AC@attglobal.net> <42173C87.4020008@cs.columbia.edu> Message-ID: <4217A08A.4070806@attglobal.net> Right. I was objecting to the use of "fluid" as a common term, because even conservation of fluid elements isn't obeyed by a protocol like TCP, or its sublayers. And, a good model has to be aware of the effects of state changes within subflows, in order to determine, even statistically, what each end will do. This particularly violates the fluid model (linear or not) when we realize the vast difference in how feedback to the originator can occur from the receiver. The bottom line in modelling is always to respect the engineer's motto: "The electrons (or whatever) know what they're doing. It's up to us to figure that out." Alex Vishal Misra wrote: > > > Cannara wrote: > >> I don't know "ignored" -- some friends (NetPredict) have a patent on >> just such >> statistical analyses applied to TCP performance products. It's the >> nonlinearity that makes any linear differential modelling grossly >> approximate. >> >> We have to remember some fundamentals -- at least that fluids are >> variously >> compressible & reorderable, unlike packets, and loss of molecules is >> not an >> option. :] >> > > "fluid" modeling doesn't imply an application of Hydraulics or > Bernoulloi's laws for protocol analysis. The "fluid" simply implies that > the quantity that is being analyzed is continuous, and not discrete. > Fluid models of TCP themselves come in two flavors > - one that analyze some mean value, and hence are intrinsically > continuous in nature (e.g., the Sigcomm 2000 paper mentioned earlier in > the thread), and, > - the other kind where the "fluid limit" or continuous property is > obtained from a scaling of some parameters to infinity, e.g., the number > of flows (the Performance 2002 paper mentioned in the same posting). > > The differential equations that are obtained are non-linear, simply > being "fluid" does not constrain them to linearity. Additionally, > stochastic differential equation based "fluid" models of TCP allow jump > discontinuities. > > -Vishal From Jon.Crowcroft at cl.cam.ac.uk Sun Feb 20 02:13:44 2005 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Sun, 20 Feb 2005 10:13:44 +0000 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: Message from Alex Cannara of "Sat, 19 Feb 2005 12:12:36 PST." <42179DB4.7090505@attglobal.net> Message-ID: dont want to argue about the non-linearity stuff - seems reasonable - but about cults and so on - i thin you hit the nail on the head (and in the research community we've bemoaned the zillion small-delta on tcp papers - last time i looked there were on the order of 3000 references on nano-deltas to TCP and acive queue management...) but i think its a bit unfair on the people that did try and think bigger: the same people fixed tcp had a very good go at various designs for fixing the ip layer if you check the history - not only are many of the same technical people associated with the better ideas in resorce reservation and fair queueing, but they also looked at getting out of the mess of IP over ATM, and tried very hard to think about _scaling_ of new network techniques for efficiency and fairness (e.g. core stateless fair queueing, and diffserv stuff that is actually in use in quite a few VOIP deployments (the ones that aren;t just freeriding on overprovisioning) the actual reasons that ip survived are non technical - they are "cant get here from there, however much you'd like to" - e.g. installed base, deployment, cost, inertia, intractable software and hardware development problems (e.g. certain router vendors in very deep potential wells:), etc etc etc hwever, this has not all been bad: see Models for a self-managed Internet F. P. Kelly http://www.statslab.cam.ac.uk/~frank/smi.html Paper presented at: Network Modelling in the 21st Century, Royal Society Discussion Meeting, London, December 1999, and published in: Philosophical Transactions of the Royal Society A358 (2000) 2335-2348. for some reasons why... and now we are still in core bandwidth glut world, but the core technology and access nets may all shift in how they work (lots of photonics in core and lots wireless at edge) we may need something that isnt like anyones vision in the last 30 years either side of the e2e v. hbh debates... I really do think there's a brick wall we are about to hit - there's huge pent up demand in end systems with nothing stopping them overloading the net except the limit on access link speed today - if you add up the 6M broadband links in the UK and take worst case assumptions, then the core ISPs and exchange point capacity is over by factors of roughly 5....if someone deployed fiber to the home at the rate they deployed cable tv coax, and then deployed just 100Mbps ether access over that, the core nets would be underprovisioned by about 2 orders of magnitude potentially in about 2 years; and no obvious upgrade path in pure technology , so paying attention to the architecture about now is probably quite a good idea the alternatives are ok in terms of all our jobs, but we will be fixing lots of stupid problems rather than off developing cool new stuff:) In missive <42179DB4.7090505 at attglobal.net>, Alex Cannara typed: >>Lordy, Lordy, since when is a threshold or timeout linear? If I drop 1% of a >>true fluid flow randomly, do I get 30% less out the other end of the pipe? >>One key to TCP's nonlinearity is its naive attempt to protect the network >>layer from congestion. The folks in the '80s who trembled with anger at >>Metcalfe's prediction of Internet collapse lived through some bad, near-miss >>scares, but instead of fixing IP and the network-layer systems, chose to make >>the transport layer the guardian of the net. This has always been a band aid, >>or kludge, depending on your upbringing, but it led to complacency and even >>religious fervor about "TCP", with cults even espousing foolery like "TCP >>Friendliness" and forclosing good ideas on doing a real fix for over a decade. >> Of course the Internet bureaucracy did deal with idiotic IPv4 addressing, >>over as many years, and did so well we have IPv6 (or we hope we never will >>have it)! >> >>{:o] >> >>Alex >> >>Saverio Mascolo wrote: >>> well this is not a technical answer! If you build a system made of linear >>> subsystems (i.e. gains, delays and integrators) how can you get a nonlinear >>> system? >>> >>> Saverio >>> >>> >>>>Well, only 15 years of modest network consulting for over 1000 modest >>>>companies from GM on down. :] >>>> >>>>Have VJ estimate how much delay is incurred by any current TCP under 1% >>> >>> random >>> >>>>packet loss; then 0.1%. Or, have him or anyone else, itemize the >>> >>> differences >>> >>>>among TCPs deployed by, say, the top 10 systems vendors. >>>> >>>>There've been many archived discussions of how TCP does & doesn't work as >>> >>> well >>> >>>>as it should, given all the years that have passed, in which continued >>>>protocol development could have, but did not, occur. Thus the Internet -- >>> >>> a >>> >>>>study in insecurity, capacity co-option, performance mediocrity, >>> >>> engineering >>> >>>>bureaucracy... But, that's not what folks want to hear. Which is why the >>>>mediocrity continues and those of us in consulting & security have good >>> >>> jobs & >>> >>>>good profits. :] >>>> >>>>Alex >>>> >>>>Saverio Mascolo wrote: >>>> >>>>>why do you think TCP congestion control is not linear? I tend to agree >>> >>> with >>> >>>>>Van Jacobson when syas that a network is to a large extend a linear >>> >>> system >>> >>>>>made of integrators, delays and gains. >>>>> >>>>>Saverio >>>>> >>>>>----- Original Message ----- >>>>>From: "Cannara" >>>>>To: >>>>>Sent: Friday, February 18, 2005 6:12 AM >>>>>Subject: Re: [e2e] link between Kelly's control and TCP's AIMD >>>>> >>>>> >>>>>>And, without understanding/modelling TCP's nonlinear behaviors, it >>> >>> will >>> >>>>>only >>>>> >>>>>>serve for gross predictions. >>>>>> >>>>>>Alex >>>>>> >>>>>>Ted Faber wrote: >>>>>> >>>>>>>On Thu, Feb 17, 2005 at 01:31:08AM +0000, Damon Wischik wrote: >>>>>>> >>>>>>>>Roy Xu wrote: >>>>>>>> >>>>>>>>>I'm looking for a pointer to literatures that link the >>>>>>>>>TCP's (discrete) AIMD to Kelly's (continuous) control >>> >>> formulation. >>> >>>>>>>>Kelly's continuous-time formulation uses a differential equation >>> >>> model >>> >>>>>>>>(also called a fluid model) for TCP. You should look at the >>> >>> literature >>> >>>>>>>>which describes this fluid model, starting with >>>>>>>> >>>>>>>>"A Fluid-based Analysis of a Network of AQM Routers Supporting TCP >>>>> >>>>>Flows >>>>> >>>>>>>>with an Application to RED", V. Misra, W. Gong, D. Towsley, >>> >>> SIGCOMM >>> >>>>>2000. >>>>> >>>>>>>Fluid flow congestion control analysis goes beck to these guys at >>> >>> least: >>> >>>>>>>%A D. Mitra >>>>>>>%A T. Seery >>>>>>>%T Dynamic Adaptive Windows for High Speed Data Networks: Theory and >>>>>>>%Simulation >>>>>>>%J Proc. SIGCOMM Symposium on Communications Architectures and >>> >>> Protocols >>> >>>>>>>%P 30-29 >>>>>>>%I ACM SIGCOMM >>>>>>>%C Philadelphia, PA >>>>>>>%D Sept 24-27, 1990 >>>>>>> >> cheers jon From cannara at attglobal.net Sun Feb 20 15:37:10 2005 From: cannara at attglobal.net (Cannara) Date: Sun, 20 Feb 2005 15:37:10 -0800 Subject: [e2e] link between Kelly's control and TCP's AIMD References: Message-ID: <42191F26.CCAFC194@attglobal.net> Right Jon. In one hot area of the network systems industry now, we see huge corporate interest in taming p2P traffic (Kazaaa, BitTorrent, eDonkey...) and IM file xfers -- not only due to traffic loading at edge interfaces, but due to copyright-suit fears and intellectual property loss (the other IP). This too is a direct result of the historically poor grasp of what's important in a truly global Internet, much as earlier issues with DOS, etc. So, the upshot is lots of $ being and to be spent on systems that analyze, report and even intervene on such flows. Most of this $ will go to companies and investors in them, but come from all of us, just as the historical subsidy for the Internet paths, protocols, conventions and other yadda yadda that's left us in the current state of mediocrity came mostly from taxpayers. What other publicly funded system has turned out to have its dominant daily service volumes in scams, spam, pornography and whatever else low lifes can think of? The Internet is its own denial of service system. The issue isn't free access, it's misappropriation -- the little secret ISPs haven't really wanted to expose -- and, the culprit? The same Internet bureacracy that cloyingly plotted graphs of doubling (or more) of variables X, Y... every year, but failed to grasp old networking truths and act on them with speed & common sense. Here's a test for any new CS grad student: Name 4 reasons the Internet is poorly designed. You have 4 minutes. :] Alex Jon Crowcroft wrote: > > dont want to argue about the non-linearity stuff - seems reasonable - > > but about cults and so on - i thin you hit the nail on the head (and in the research community > we've bemoaned the zillion small-delta on tcp papers - last time i looked there were on the order of > 3000 references on nano-deltas to TCP and acive queue management...) > > but i think its a bit unfair on the people that did try and think bigger: > the same people fixed tcp had a very good go at various designs for > fixing the ip layer if you check the history - not only are many of the same > technical people associated with the better ideas in resorce reservation > and fair queueing, but they also looked at getting out of the mess of IP over ATM, > and tried very hard to think about _scaling_ of new network techniques for efficiency > and fairness (e.g. core stateless fair queueing, and diffserv stuff that is actually > in use in quite a few VOIP deployments (the ones that aren;t just freeriding on overprovisioning) > > the actual reasons that ip survived are non technical - they are "cant get here from there, however much you'd like > to" - e.g. installed base, deployment, cost, inertia, intractable software and hardware development problems > (e.g. certain router vendors in very deep potential wells:), etc etc etc > > hwever, this has not all been bad: > see > Models for a self-managed Internet > F. P. Kelly > http://www.statslab.cam.ac.uk/~frank/smi.html > Paper presented at: > Network Modelling in the 21st Century, > Royal Society Discussion Meeting, > London, December 1999, and published in: > Philosophical Transactions of the Royal Society A358 (2000) 2335-2348. > for some reasons why... > > and now we are still in core bandwidth glut world, but the core technology and access nets may all shift in how > they work (lots of photonics in core and lots wireless at edge) we may need something that isnt like anyones vision > in the last 30 years either side of the e2e v. hbh debates... > > I really do think there's a brick wall we are about to hit - there's huge pent up demand in end systems with > nothing stopping them overloading the net except the limit on access link speed today - if you add up the 6M > broadband links in the UK and take worst case assumptions, then the core ISPs and exchange point capacity is over > by factors of roughly 5....if someone deployed fiber to the home at the rate they deployed cable tv coax, and then > deployed just 100Mbps ether access over that, the core nets would be underprovisioned by about 2 orders of > magnitude potentially in about 2 years; and no obvious upgrade path in pure technology , so paying attention > to the architecture about now is probably quite a good idea > > the alternatives are ok in terms of all our jobs, but we will be fixing lots of stupid problems rather than off > developing cool new stuff:) > > In missive <42179DB4.7090505 at attglobal.net>, Alex Cannara typed: > > >>Lordy, Lordy, since when is a threshold or timeout linear? If I drop 1% of a > >>true fluid flow randomly, do I get 30% less out the other end of the pipe? > >>One key to TCP's nonlinearity is its naive attempt to protect the network > >>layer from congestion. The folks in the '80s who trembled with anger at > >>Metcalfe's prediction of Internet collapse lived through some bad, near-miss > >>scares, but instead of fixing IP and the network-layer systems, chose to make > >>the transport layer the guardian of the net. This has always been a band aid, > >>or kludge, depending on your upbringing, but it led to complacency and even > >>religious fervor about "TCP", with cults even espousing foolery like "TCP > >>Friendliness" and forclosing good ideas on doing a real fix for over a decade. > >> Of course the Internet bureaucracy did deal with idiotic IPv4 addressing, > >>over as many years, and did so well we have IPv6 (or we hope we never will > >>have it)! > >> > >>{:o] > >> > >>Alex > >> > >>Saverio Mascolo wrote: > >>> well this is not a technical answer! If you build a system made of linear > >>> subsystems (i.e. gains, delays and integrators) how can you get a nonlinear > >>> system? > >>> > >>> Saverio > >>> > >>> > >>>>Well, only 15 years of modest network consulting for over 1000 modest > >>>>companies from GM on down. :] > >>>> > >>>>Have VJ estimate how much delay is incurred by any current TCP under 1% > >>> > >>> random > >>> > >>>>packet loss; then 0.1%. Or, have him or anyone else, itemize the > >>> > >>> differences > >>> > >>>>among TCPs deployed by, say, the top 10 systems vendors. > >>>> > >>>>There've been many archived discussions of how TCP does & doesn't work as > >>> > >>> well > >>> > >>>>as it should, given all the years that have passed, in which continued > >>>>protocol development could have, but did not, occur. Thus the Internet -- > >>> > >>> a > >>> > >>>>study in insecurity, capacity co-option, performance mediocrity, > >>> > >>> engineering > >>> > >>>>bureaucracy... But, that's not what folks want to hear. Which is why the > >>>>mediocrity continues and those of us in consulting & security have good > >>> > >>> jobs & > >>> > >>>>good profits. :] > >>>> > >>>>Alex > >>>> > >>>>Saverio Mascolo wrote: > >>>> > >>>>>why do you think TCP congestion control is not linear? I tend to agree > >>> > >>> with > >>> > >>>>>Van Jacobson when syas that a network is to a large extend a linear > >>> > >>> system > >>> > >>>>>made of integrators, delays and gains. > >>>>> > >>>>>Saverio > >>>>> > >>>>>----- Original Message ----- > >>>>>From: "Cannara" > >>>>>To: > >>>>>Sent: Friday, February 18, 2005 6:12 AM > >>>>>Subject: Re: [e2e] link between Kelly's control and TCP's AIMD > >>>>> > >>>>> > >>>>>>And, without understanding/modelling TCP's nonlinear behaviors, it > >>> > >>> will > >>> > >>>>>only > >>>>> > >>>>>>serve for gross predictions. > >>>>>> > >>>>>>Alex > >>>>>> > >>>>>>Ted Faber wrote: > >>>>>> > >>>>>>>On Thu, Feb 17, 2005 at 01:31:08AM +0000, Damon Wischik wrote: > >>>>>>> > >>>>>>>>Roy Xu wrote: > >>>>>>>> > >>>>>>>>>I'm looking for a pointer to literatures that link the > >>>>>>>>>TCP's (discrete) AIMD to Kelly's (continuous) control > >>> > >>> formulation. > >>> > >>>>>>>>Kelly's continuous-time formulation uses a differential equation > >>> > >>> model > >>> > >>>>>>>>(also called a fluid model) for TCP. You should look at the > >>> > >>> literature > >>> > >>>>>>>>which describes this fluid model, starting with > >>>>>>>> > >>>>>>>>"A Fluid-based Analysis of a Network of AQM Routers Supporting TCP > >>>>> > >>>>>Flows > >>>>> > >>>>>>>>with an Application to RED", V. Misra, W. Gong, D. Towsley, > >>> > >>> SIGCOMM > >>> > >>>>>2000. > >>>>> > >>>>>>>Fluid flow congestion control analysis goes beck to these guys at > >>> > >>> least: > >>> > >>>>>>>%A D. Mitra > >>>>>>>%A T. Seery > >>>>>>>%T Dynamic Adaptive Windows for High Speed Data Networks: Theory and > >>>>>>>%Simulation > >>>>>>>%J Proc. SIGCOMM Symposium on Communications Architectures and > >>> > >>> Protocols > >>> > >>>>>>>%P 30-29 > >>>>>>>%I ACM SIGCOMM > >>>>>>>%C Philadelphia, PA > >>>>>>>%D Sept 24-27, 1990 > >>>>>>> > >> > > cheers > > jon From braden at ISI.EDU Sun Feb 20 17:59:34 2005 From: braden at ISI.EDU (Bob Braden) Date: Sun, 20 Feb 2005 17:59:34 -0800 (PST) Subject: [e2e] link between Kelly's control and TCP's AIMD Message-ID: <200502210159.RAA09294@gra.isi.edu> Mr. Cannara, Your message contains the words: naive, kludge, religous fervor, foolery, bureaucracy, and idiotic. You are entitled to your opinions, but you are not entitled to pollute this mailing list with messages of this acid flavor. I have warned you about this before. Bob Braden From arjuna at dcs.kcl.ac.uk Mon Feb 21 03:11:06 2005 From: arjuna at dcs.kcl.ac.uk (Arjuna Sathiaseelan) Date: Mon, 21 Feb 2005 11:11:06 -0000 (GMT) Subject: [e2e] Routers accessing TCP header Message-ID: <64196.137.73.8.3.1108984266.squirrel@137.73.8.3> Dear all, I have implemented a mechanism which requires routers to access the TCP header from the IP packet mainly the TCP sequence numbers. Implementing this on ns-2 is not a problem. But in real world, is this a possibility? Can the routers access the TCP header mainly the TCP sequence numbers? I would be very much obliged if someone could help me with this. Regards, Arjuna From Jon.Crowcroft at cl.cam.ac.uk Mon Feb 21 03:53:19 2005 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Mon, 21 Feb 2005 11:53:19 +0000 Subject: [e2e] Routers accessing TCP header In-Reply-To: Message from "Arjuna Sathiaseelan" of "Mon, 21 Feb 2005 11:11:06 GMT." <64196.137.73.8.3.1108984266.squirrel@137.73.8.3> Message-ID: in principle, anything is possible given enough money... in practice, depends on the router deep packet inspection takes lotsa cycles or usesup valuble logic in any custom processor stuff on line cards, or means your packet goes to the control procxessor ("slow path") but also messes up any router that has seperate paths for the headers or any router with a switch that turns packets into cells etc etc etc will all make life very hard if its a zebra/xorp/gated or any other unix variant, then its probably about as easy as changing ns...and about as reliable:) depends on the packet if its crypted, you lose Hack proposal: copy the TCP sequence number into the IP Id field then its in the normal IP header (yes i know the TCP sequence number is 32 but, but it usually varies by a MSS size, and you could just put the deltas in the IP field - surely noone depends on them going ++ (they'd stil be _monotocally_ increaseing, just less monotonously:) and the ip frag stuff would still all work right? In missive <64196.137.73.8.3.1108984266.squirrel at 137.73.8.3>, "Arjuna Sathiaseelan" typed: >> >> >>Dear all, >> I have implemented a mechanism which requires routers to access the TCP >>header from the IP packet mainly the TCP sequence numbers. Implementing >>this on ns-2 is not a problem. But in real world, is this a possibility? >>Can the routers access the TCP header mainly the TCP sequence numbers? I >>would be very much obliged if someone could help me with this. >> >>Regards, >>Arjuna >> >> cheers jon From arjuna at dcs.kcl.ac.uk Mon Feb 21 04:19:16 2005 From: arjuna at dcs.kcl.ac.uk (Arjuna Sathiaseelan) Date: Mon, 21 Feb 2005 12:19:16 -0000 (GMT) Subject: [e2e] Routers accessing TCP header Message-ID: <41004.137.73.8.3.1108988356.squirrel@137.73.8.3> Dear Jon, Thanks for your appropriate reply. I am basically working on a mechanism called the Explicit Packet Drop Notification (EPDN). EPDN works like this : Every router has a hashtable which stores the flow id and the maximum and minimum sequence numbers of the dropped packets. This information will then be inserted into a packet of that particular flow that passes through the gateway successfully. The receiver in turn runs a series of algorithms to detect whether a missing packet is dropped or not. Based on this the receiver informs the sender about the cause of the inorder packet. We do not maintain per flow state - just the state of flows of dropped packets. Now this mechanism infact has lots of limitations such as not incremental and requires all the gateways to be implemented with the EPDN mechanism and as you said requires lots of CPU cycles for deep packet inspection. Do you think this type of mechanism has any future in real world implementation and do you feel that maintaining the max and min sequence numbers of dropped packets is a possibility given that there is IP fragmentation going on? Your suggestions will be really helpful.. Regards, Arjuna From baruch at ev-en.org Mon Feb 21 06:08:52 2005 From: baruch at ev-en.org (Baruch Even) Date: Mon, 21 Feb 2005 14:08:52 +0000 Subject: [e2e] [Fwd: BicTCP Implementation Bug] Message-ID: <4219EB74.7020206@ev-en.org> This was posted by a colleague who is not subscribed to the list. Baruch -------------- next part -------------- An embedded message was scrubbed... From: Yee-Ting Li Subject: BicTCP Implementation Bug Date: Mon, 21 Feb 2005 13:47:29 +0000 Size: 3224 Url: http://www.postel.org/pipermail/end2end-interest/attachments/20050221/db3e1f79/BicTCPImplementationBug.mht From dpreed at reed.com Mon Feb 21 07:31:00 2005 From: dpreed at reed.com (David P. Reed) Date: Mon, 21 Feb 2005 10:31:00 -0500 Subject: [e2e] Routers accessing TCP header In-Reply-To: References: Message-ID: <4219FEB4.5080109@reed.com> Arjuna - it's worth noting the earlier discussion about fields shared between TCP and the IP layer (and the cost imposed on the end-to-end protocol's properties, such as security, etc.) You can put anything in the IP layer you want, including a copy of the TCP sequence number. [or living dangerously, "share" the sequence number between layers as you propose]. However, I'd question why you are not solving the more general problem - for example supporting RTP, which also has "sequence number like" fields, and which ought to be doing congestion control. Why not just extend IP to include a non-decreasing number that indicates progress to the router? Lazy TCP implementers can just use the TCP Sequence number for that field, RTP can use the frame number of the video or audio, etc. Encrypted protocols could use a non-decreasing sequence number of their own devising, perhaps structured to avoid unnecessary exposure of application progress (for example each retransmitted packet could have a higher sequence number, so that the man-in-the-middle isn't able to use forcing of retransmit to determine if the encrypted protocol is retransmission-oriented, thereby distinguishing TCP from RTP by using responses to stimuli). That would be a forward-looking contribution to protocol-independent networking, rather than yet another kludge that presumes the IP layer should be able to read all traffic. From faber at ISI.EDU Mon Feb 21 10:07:02 2005 From: faber at ISI.EDU (Ted Faber) Date: Mon, 21 Feb 2005 10:07:02 -0800 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: <4216F217.647A24AC@attglobal.net> References: <000a01c50f95$98bfd660$f01aeb80@snow> <20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> <20050218181852.GO90698@pun.isi.edu> <4216F217.647A24AC@attglobal.net> Message-ID: <20050221180702.GD16091@pun.isi.edu> On Sat, Feb 19, 2005 at 12:00:23AM -0800, Cannara wrote: > I don't know "ignored" -- some friends (NetPredict) have a patent on just such > statistical analyses applied to TCP performance products. It's the > nonlinearity that makes any linear differential modelling grossly approximate. > > We have to remember some fundamentals -- at least that fluids are variously > compressible & reorderable, unlike packets, and loss of molecules is not an > option. :] I was going to ask if you'd even read the papers. Thanks for answering that question. -- Ted Faber http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.asc Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050221/24dc8384/attachment-0001.bin From cannara at attglobal.net Mon Feb 21 19:31:16 2005 From: cannara at attglobal.net (Cannara) Date: Mon, 21 Feb 2005 19:31:16 -0800 Subject: [e2e] link between Kelly's control and TCP's AIMD References: <000a01c50f95$98bfd660$f01aeb80@snow> <20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> <20050218181852.GO90698@pun.isi.edu> <4216F217.647A24AC@attglobal.net> <20050221180702.GD16091@pun.isi.edu> Message-ID: <421AA784.1BB0A1CC@attglobal.net> Ooooh Ted, right through the heart! Other papers I have read on similar TCP performance modelling have had some interesting errors, later admitted to, but I promise to take some time to check the ones you suggest. Alex Ted Faber wrote: > > On Sat, Feb 19, 2005 at 12:00:23AM -0800, Cannara wrote: > > I don't know "ignored" -- some friends (NetPredict) have a patent on just such > > statistical analyses applied to TCP performance products. It's the > > nonlinearity that makes any linear differential modelling grossly approximate. > > > > We have to remember some fundamentals -- at least that fluids are variously > > compressible & reorderable, unlike packets, and loss of molecules is not an > > option. :] > > I was going to ask if you'd even read the papers. Thanks for answering > that question. > > -- > Ted Faber > http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.asc > Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG > > ------------------------------------------------------------------------------ > Part 1.2Type: application/pgp-signature From cannara at attglobal.net Mon Feb 21 19:40:07 2005 From: cannara at attglobal.net (Cannara) Date: Mon, 21 Feb 2005 19:40:07 -0800 Subject: [e2e] link between Kelly's control and TCP's AIMD References: <200502210159.RAA09294@gra.isi.edu> Message-ID: <421AA997.76D9D27@attglobal.net> Did I really mis-spell "religous" Bob? Well, at least it didn't cost anyone millions of bucks on anti-spam gear. Oops, is "spam" polluting? Alex (trembling in Silicon Valley) Bob Braden wrote: > > Mr. Cannara, > > Your message contains the words: naive, kludge, religous fervor, > foolery, bureaucracy, and idiotic. You are entitled to your opinions, > but you are not entitled to pollute this mailing list with messages of > this acid flavor. > > I have warned you about this before. > > Bob Braden > From Jon.Crowcroft at cl.cam.ac.uk Tue Feb 22 00:25:57 2005 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Tue, 22 Feb 2005 08:25:57 +0000 Subject: [e2e] Routers accessing TCP header In-Reply-To: Message from "Arjuna Sathiaseelan" of "Mon, 21 Feb 2005 12:19:16 GMT." <41004.137.73.8.3.1108988356.squirrel@137.73.8.3> Message-ID: so if a router is dropping packets beause of queues being full then for EPDN to work reasoanbly well, you are going to end up keeping on the order of the number of flows worth of state (although the amount of state aint that much it will at least be the 5 tuple to identify flow, plus sequence number so 16 bytes or more...) the rate of access to this wil ldepend on the nature of the EPDN mech - e.g. if its like ECN, you only need to mark a flow with binary feedback, but if its part of the +reliability+ scheme, you need to correctly "nack" each missed packet...so that ends up being quite a lot of processing... I guess the assumption is that packets either get delivered end to end, with or without ECN (early warning) or they are knowingly dropped (EPDN) or lost without knowledge (need to rely on fast recovery etc)... so i suppose if ECN is operating right, EPDN would be rare...but without ECN, or with end systems that dont respond to ECN right, EPDN is going to dominate - and thats not going to be sustainable unless its in the edge router (and it might be as most ISPs claim there's little congestion if any in the core and most IXPs now claim there's little congestion in the Internet Exchanges either (certainly true at the LINX where i've seen the figures:) In missive <41004.137.73.8.3.1108988356.squirrel at 137.73.8.3>, "Arjuna Sathiaseelan" typed: >> Thanks for your appropriate reply. I am basically working on a mechanism >>called the Explicit Packet Drop Notification (EPDN). EPDN works like >>this : Every router has a hashtable which stores the flow id and the >>maximum and minimum sequence numbers of the dropped packets. This >>information will then be inserted into a packet of that particular flow >>that passes through the gateway successfully. The receiver in turn runs >>a series of algorithms to detect whether a missing packet is dropped or >>not. Based on this the receiver informs the sender about the cause of >>the inorder packet. We do not maintain per flow state - just the state >>of flows of dropped packets. >> >> >>Now this mechanism infact has lots of limitations such as not incremental >>and requires all the gateways to be implemented with the EPDN mechanism >>and as you said requires lots of CPU cycles for deep packet inspection. >> >>Do you think this type of mechanism has any future in real world >>implementation and do you feel that maintaining the max and min sequence >>numbers of dropped packets is a possibility given that there is IP >>fragmentation going on? >> >>Your suggestions will be really helpful.. >> >>Regards, >>Arjuna >> >> cheers jon From tvest at pch.net Tue Feb 22 01:09:07 2005 From: tvest at pch.net (Tom Vest) Date: Tue, 22 Feb 2005 18:09:07 +0900 Subject: [e2e] Measuring the benefits of statistical multiplexing Message-ID: Would be grateful if someone could point me to books/papers that attempt to empirically measure the benefits of statistical multiplexing.... Thanks in advance, Tom From s.malik at tuhh.de Tue Feb 22 02:28:22 2005 From: s.malik at tuhh.de (Sireen Habib Malik) Date: Tue, 22 Feb 2005 11:28:22 +0100 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: <421AA784.1BB0A1CC@attglobal.net> References: <000a01c50f95$98bfd660$f01aeb80@snow> <20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> <20050218181852.GO90698@pun.isi.edu> <4216F217.647A24AC@attglobal.net> <20050221180702.GD16091@pun.isi.edu> <421AA784.1BB0A1CC@attglobal.net> Message-ID: <421B0946.1000406@tuhh.de> Hi, I just want to continue on the comment that "(tcp) performance modelling have had some interesting errors". I take this as a very good chance to understand a few things here. Following is the most basic model (I am starting with the abc... which is a very nice place to start). Area under the curve assuming periodic "packet loss" (AIMD control) a. total_packets = (w*w)/ 4+ (w*w)/8 = 3*w*w/8 b. loss probability (p) = (1 lost packet) /(total_packets) = 8/(3*w*w) Therefore, w= sqrt (8/(3*p). Since average_window = (3*w)/4, therefore, average_window = sqrt (1.5/p). Risking the nausea of many people here, i have included all the trivial steps. This very straight forward approach does lead to a non-linear relationship! What are the errors in this approach? I am sure that knowing the problem with this one will lead to a deeper understanding of the "Troubled" Control Protocol (excuse my twist but the preceding discussions do give the impression!) Thank you for the response. Sireen Malik TUHH Hamburg, Gemany Cannara wrote: >Ooooh Ted, right through the heart! Other papers I have read on similar TCP >C later admitted to, but >I promise to take some time to check the ones you suggest. > >Alex > >Ted Faber wrote: > > >>On Sat, Feb 19, 2005 at 12:00:23AM -0800, Cannara wrote: >> >> >>>I don't know "ignored" -- some friends (NetPredict) have a patent on just such >>>statistical analyses applied to TCP performance products. It's the >>>nonlinearity that makes any linear differential modelling grossly approximate. >>> >>>We have to remember some fundamentals -- at least that fluids are variously >>>compressible & reorderable, unlike packets, and loss of molecules is not an >>>option. :] >>> >>> >>I was going to ask if you'd even read the papers. Thanks for answering >>that question. >> >>-- >>Ted Faber >>http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.asc >>Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG >> >> ------------------------------------------------------------------------------ >> Part 1.2Type: application/pgp-signature >> >> -- Sireen Malik, M.Sc. PhD. Candidate, Communication Networks Hamburg University of Technology, FSP 4-06 (room 3008) Denickestr. 17 21073 Hamburg, Deutschland Tel: +49 (40) 42-878-3387 Fax: +49 (40) 42-878-2941 E-Mail: s.malik at tuhh.de --Everything should be as simple as possible, but no simpler (Albert Einstein) From fla at inescporto.pt Tue Feb 22 06:17:55 2005 From: fla at inescporto.pt (Filipe Abrantes) Date: Tue, 22 Feb 2005 14:17:55 +0000 Subject: [e2e] Papers on router performance Message-ID: <421B3F13.2000306@inescporto.pt> Does someone know any papers or practical studies on router (hardware) performance limitations - specially about its performance when they have to examine certain fields on IP or network header on a per-packet basis. For instance, what is the maximum output bandwidth when it has to examine the TOS field in each incoming packet. Hope I could make my self clear and Thanks for your help Filipe -- Filipe Lameiro Abrantes INESC Porto Campus da FEUP Rua Dr. Roberto Frias, 378 4200-465 Porto Portugal Phone: +351 22 209 4266 E-mail: fla at inescporto.pt From christin at SIMS.Berkeley.EDU Tue Feb 22 07:55:58 2005 From: christin at SIMS.Berkeley.EDU (Nicolas Christin) Date: Tue, 22 Feb 2005 07:55:58 -0800 Subject: [e2e] Internet Congestion (Was: Re: Routers accessing TCP header) In-Reply-To: References: <41004.137.73.8.3.1108988356.squirrel@137.73.8.3> Message-ID: <20050222155558.GA7127@dogmatix.SIMS.Berkeley.EDU> [Snipping a very interesting discussion to ask a specific question] On Tue Feb 22, 2005, Jon Crowcroft wrote: > > most IXPs now claim there's little congestion in the > Internet Exchanges either (certainly true at the LINX where i've seen > the figures:) Jon, That is actually something that I have been wondering about for quite a while. Is there any published report or paper that substantiates this claim with numbers? I believe it. I just would like to see the numbers as well :) Best, -- Nicolas -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050222/0f342c4c/attachment.bin From craig at aland.bbn.com Tue Feb 22 09:08:38 2005 From: craig at aland.bbn.com (Craig Partridge) Date: Tue, 22 Feb 2005 12:08:38 -0500 Subject: [e2e] Routers accessing TCP header In-Reply-To: Your message of "Mon, 21 Feb 2005 12:19:16 GMT." <41004.137.73.8.3.1108988356.squirrel@137.73.8.3> Message-ID: <20050222170838.DF18B243@aland.bbn.com> In message <41004.137.73.8.3.1108988356.squirrel at 137.73.8.3>, "Arjuna Sathiasee lan" writes: > >Dear Jon, > Thanks for your appropriate reply. I am basically working on a mechanism >called the Explicit Packet Drop Notification (EPDN). While the mechanism is slightly different, this sounds like Explicit Transport Error Notification (ETEN) done again. See: Rajesh Krishnan, James P.G. Sterbenz, Wesley M. Eddy, Craig Partridge and Mark Allman, "Explicit transport error notification (ETEN) for error-prone wireless and satellite networks," Computer Networks, 46(3), October 2004, pp. 343-362. I'll note that my co-authors and I continue to debate exactly how much benefit such a service provides. My reaction, from the "Oracle ETEN" simulations in the paper (where we report the maximum improvement possible if you know perfectly and immediately which packets are lost) is that the gains are not enough to merit implementing the service. Your mileage may vary! Craig From craig at aland.bbn.com Tue Feb 22 09:22:42 2005 From: craig at aland.bbn.com (Craig Partridge) Date: Tue, 22 Feb 2005 12:22:42 -0500 Subject: [e2e] Papers on router performance In-Reply-To: Your message of "Tue, 22 Feb 2005 14:17:55 GMT." <421B3F13.2000306@inescporto.pt> Message-ID: <20050222172242.C219D243@aland.bbn.com> Hi: This is a fun question but you have to refine it. If you are saying, "what can be done on current CISCO/JUNIPER hardware" I don't know of any papers, but I know lots of folks who can probably answer the question. If you are asking, "what could routers do?" the answer is, generally, not an absolute, but rather a matter of "how much money do you have?" In general, hardware can be built to run at line rate, and then you simply build deeper and deeper pipelines to fit in all the functionality you want... Craig In message <421B3F13.2000306 at inescporto.pt>, Filipe Abrantes writes: >Does someone know any papers or practical studies on router (hardware) >performance limitations - specially about its performance when they have >to examine certain fields on IP or network header on a per-packet basis. > >For instance, what is the maximum output bandwidth when it has to >examine the TOS field in each incoming packet. > >Hope I could make my self clear and >Thanks for your help > >Filipe > >-- >Filipe Lameiro Abrantes >INESC Porto >Campus da FEUP >Rua Dr. Roberto Frias, 378 >4200-465 Porto >Portugal > >Phone: +351 22 209 4266 >E-mail: fla at inescporto.pt From salehi at cisco.com Tue Feb 22 09:36:58 2005 From: salehi at cisco.com (Nader Salehi) Date: Tue, 22 Feb 2005 09:36:58 -0800 Subject: [e2e] Papers on router performance In-Reply-To: <20050222172242.C219D243@aland.bbn.com> References: <421B3F13.2000306@inescporto.pt> <20050222172242.C219D243@aland.bbn.com> Message-ID: <16923.28090.873730.173803@localhost.localdomain> Craig Partridge writes: > > Hi: > > This is a fun question but you have to refine it. > > If you are saying, "what can be done on current CISCO/JUNIPER hardware" > I don't know of any papers, but I know lots of folks who can probably > answer the question. > You might want to take a look at the first and second ACM SIGCOMM Internet Measurement Workshops. In them you might find some performance measurements on various routers. Good luck! Nader > If you are asking, "what could routers do?" the answer is, generally, not > an absolute, but rather a matter of "how much money do you have?" In > general, hardware can be built to run at line rate, and then you simply > build deeper and deeper pipelines to fit in all the functionality you > want... > > Craig > > In message <421B3F13.2000306 at inescporto.pt>, Filipe Abrantes writes: > > >Does someone know any papers or practical studies on router (hardware) > >performance limitations - specially about its performance when they have > >to examine certain fields on IP or network header on a per-packet basis. > > > >For instance, what is the maximum output bandwidth when it has to > >examine the TOS field in each incoming packet. > > > >Hope I could make my self clear and > >Thanks for your help > > > >Filipe > > > >-- > >Filipe Lameiro Abrantes > >INESC Porto > >Campus da FEUP > >Rua Dr. Roberto Frias, 378 > >4200-465 Porto > >Portugal > > > >Phone: +351 22 209 4266 > >E-mail: fla at inescporto.pt From krash at bbn.com Tue Feb 22 11:54:56 2005 From: krash at bbn.com (Rajesh Krishnan) Date: Tue, 22 Feb 2005 14:54:56 -0500 (EST) Subject: [e2e] Routers accessing TCP header In-Reply-To: <20050222170838.DF18B243@aland.bbn.com> from "Craig Partridge" at Feb 22, 2005 12:08:38 PM Message-ID: <200502221954.j1MJsue00909@a.bbn.com> > >Dear Jon, > > Thanks for your appropriate reply. I am basically working on a mechanism > >called the Explicit Packet Drop Notification (EPDN). > > While the mechanism is slightly different, this sounds like Explicit > Transport Error Notification (ETEN) done again. See: > > Rajesh Krishnan, James P.G. Sterbenz, Wesley M. Eddy, Craig Partridge > and Mark Allman, "Explicit transport error notification (ETEN) for > error-prone wireless and satellite networks," Computer Networks, > 46(3), October 2004, pp. 343-362. > > I'll note that my co-authors and I continue to debate exactly how much > benefit such a service provides. My reaction, from the "Oracle ETEN" > simulations in the paper (where we report the maximum improvement possible > if you know perfectly and immediately which packets are lost) is that > the gains are not enough to merit implementing the service. > Your mileage may vary! > > Craig The Oracle ETEN plots do show that there are benefits from notifications at _high_ error rates. The debate is about: - are the high error rates at which we see appreciable gains from notifications just too disruptive to be usable in practice (e.g. even for neighbor discovery and routing protocols to function properly, let alone run TCP) - are the environments that can benefit from notifications just too special to merit retrofitting a general-purpose transport protocol (and would it not be more efficient to handle these special anomalies locally rather than end-to-end) Notifications on a per-packet basis consume additional network resources and are potentially a security risk as well; schemes that can work with cumulative notifications are desirable. Unfortunately the performance of the cumulative scheme we developed was nowhere near as good as the Oracle ETEN results. This is not to say that better schemes cannot be devised (and I believe they can), but a moot point if you agree with Craig that there is not much benefit to be realized to begin with even with perfect instantaneous knowledge of losses. Best Regards, Rajesh From robertk at video-phones-evdo.com Tue Feb 22 12:45:00 2005 From: robertk at video-phones-evdo.com (Robert Kim, Wireless Internet / Wifi Hotspot Advisor) Date: Tue, 22 Feb 2005 12:45:00 -0800 Subject: [e2e] remove In-Reply-To: <200502221954.j1MJsue00909@a.bbn.com> Message-ID: X---------------- Robert Kim, Wireless Internet Wifi Hotspot Advisor http://evdo-coverage.com http://wireless-internet-broadband-service.com https://evdo.sslpowered.com/wifi-hotspot-router.htm 2611 S Pacific Coast Highway 101 Cardiff by the Sea CA 92007 : 206 984 0880 >>> "Wireless Internet Service Is ONLY Broadband with Broadband Customer Service"(tm) >>> OUR QUEST: To Kill the Cubicle! (SM) ---Shalommmmmmmm-------------------- ---------------------------------;-)---- -----Original Message----- From: end2end-interest-bounces at postel.org [mailto:end2end-interest-bounces at postel.org] On Behalf Of Rajesh Krishnan Sent: Tuesday, February 22, 2005 11:55 AM To: Craig Partridge Cc: end2end-interest at postel.org; Arjuna Sathiaseelan Subject: Re: [e2e] Routers accessing TCP header > >Dear Jon, > > Thanks for your appropriate reply. I am basically working on a > >mechanism called the Explicit Packet Drop Notification (EPDN). > > While the mechanism is slightly different, this sounds like Explicit > Transport Error Notification (ETEN) done again. See: > > Rajesh Krishnan, James P.G. Sterbenz, Wesley M. Eddy, Craig Partridge > and Mark Allman, "Explicit transport error notification (ETEN) for > error-prone wireless and satellite networks," Computer Networks, > 46(3), October 2004, pp. 343-362. > > I'll note that my co-authors and I continue to debate exactly how much > benefit such a service provides. My reaction, from the "Oracle ETEN" > simulations in the paper (where we report the maximum improvement > possible if you know perfectly and immediately which packets are lost) > is that the gains are not enough to merit implementing the service. > Your mileage may vary! > > Craig The Oracle ETEN plots do show that there are benefits from notifications at _high_ error rates. The debate is about: - are the high error rates at which we see appreciable gains from notifications just too disruptive to be usable in practice (e.g. even for neighbor discovery and routing protocols to function properly, let alone run TCP) - are the environments that can benefit from notifications just too special to merit retrofitting a general-purpose transport protocol (and would it not be more efficient to handle these special anomalies locally rather than end-to-end) Notifications on a per-packet basis consume additional network resources and are potentially a security risk as well; schemes that can work with cumulative notifications are desirable. Unfortunately the performance of the cumulative scheme we developed was nowhere near as good as the Oracle ETEN results. This is not to say that better schemes cannot be devised (and I believe they can), but a moot point if you agree with Craig that there is not much benefit to be realized to begin with even with perfect instantaneous knowledge of losses. Best Regards, Rajesh From arjuna.sathiaseelan at gmail.com Wed Feb 23 00:53:53 2005 From: arjuna.sathiaseelan at gmail.com (Arjuna Sathiaseelan) Date: Wed, 23 Feb 2005 08:53:53 +0000 Subject: [e2e] Routers accessing TCP header In-Reply-To: <200502221954.j1MJsue00909@a.bbn.com> References: <20050222170838.DF18B243@aland.bbn.com> <200502221954.j1MJsue00909@a.bbn.com> Message-ID: <1ef22590050223005339b9c079@mail.gmail.com> Dear Craig, Thanks for your reply. > While the mechanism is slightly different, this sounds like Explicit > Transport Error Notification (ETEN) done again. See: > > Rajesh Krishnan, James P.G. Sterbenz, Wesley M. Eddy, Craig Partridge > and Mark Allman, "Explicit transport error notification (ETEN) for > error-prone wireless and satellite networks," Computer Networks, > 46(3), October 2004, pp. 343-362. > Incase you assume that EPDN was inspired from ETEN :), This mechanism is a part of my four year PhD Thesis and has evolved through these publications : 1) Robust TCP (TCP-R) with Explicit Packet Drop Notification (EPDN) for Satellite Networks. To appear in the proceedings of the 4th International Conference on Networking (ICN '05), Reunion Island. Arjuna Sathiaseelan, Tomasz Radzik 2) Improving the Performance of TCP in the Case of Packet Reordering, 7th IEEE International Conference on High Speed Networks and Multimedia Communications HSNMC'04,Toulouse, France July 2004 (LNCS 3079, pp. 63-75). 3) RD-TCP: Reorder Detecting TCP. In proceedings of the 6th IEEE International Conference on High Speed Networks and Multimedia Communications HSNMC'03, Estoril, Portugal, July 2003 (LNCS 2720, pp.471-480). I use the EPDN mechanisms for detecting packet losses and thus distinguish a packet reorder event from a packet drop event for all these 3 papers. All these papers were products of a NOVICE UNSUPERVISED PhD work and I am in the process of improving things. I would also like to tell you frankly that EPDN is just an amateur while compared to your ETEN and no where near yours :).. I will be putting my papers on the web as soon as I get them done properly. At present they look quite amusing to me :).. Pl do not mistake this email. Its just for your information. Thanks for your reply once again. Regards, Arjuna From matta at cs.bu.edu Wed Feb 23 08:37:59 2005 From: matta at cs.bu.edu (Ibrahim Matta) Date: Wed, 23 Feb 2005 11:37:59 -0500 Subject: [e2e] Routers accessing TCP header Message-ID: <0511C607B17F804EBE96FFECD1FD98591E829B@cs-exs2.cs-nt.bu.edu> I admit not reading this paper (yet:) But I am wondering what kind of recovery strategy is used (e.g. TCP not backing off completely, or...) once you know the "exact" reason of loss (through your oracle). I think how much gain will depend not only on the "binary" classification of loss type (e.g. buffer overflow or transmission error) but also on the action TCP flows take, how synchronized they are, etc. Actually, a finer grained error classification (e.g. short-term vs. long-term error conditions etc.) along with a finer grained recovery response maybe needed? Best, Ibrahim -----Original Message----- From: end2end-interest-bounces at postel.org [mailto:end2end-interest-bounces at postel.org] On Behalf Of Rajesh Krishnan Sent: Tuesday, February 22, 2005 2:55 PM To: Craig Partridge Cc: end2end-interest at postel.org; Arjuna Sathiaseelan Subject: Re: [e2e] Routers accessing TCP header > >Dear Jon, > > Thanks for your appropriate reply. I am basically working on a > >mechanism called the Explicit Packet Drop Notification (EPDN). > > While the mechanism is slightly different, this sounds like Explicit > Transport Error Notification (ETEN) done again. See: > > Rajesh Krishnan, James P.G. Sterbenz, Wesley M. Eddy, Craig Partridge > and Mark Allman, "Explicit transport error notification (ETEN) for > error-prone wireless and satellite networks," Computer Networks, > 46(3), October 2004, pp. 343-362. > > I'll note that my co-authors and I continue to debate exactly how much > benefit such a service provides. My reaction, from the "Oracle ETEN" > simulations in the paper (where we report the maximum improvement > possible if you know perfectly and immediately which packets are lost) > is that the gains are not enough to merit implementing the service. > Your mileage may vary! > > Craig The Oracle ETEN plots do show that there are benefits from notifications at _high_ error rates. The debate is about: - are the high error rates at which we see appreciable gains from notifications just too disruptive to be usable in practice (e.g. even for neighbor discovery and routing protocols to function properly, let alone run TCP) - are the environments that can benefit from notifications just too special to merit retrofitting a general-purpose transport protocol (and would it not be more efficient to handle these special anomalies locally rather than end-to-end) Notifications on a per-packet basis consume additional network resources and are potentially a security risk as well; schemes that can work with cumulative notifications are desirable. Unfortunately the performance of the cumulative scheme we developed was nowhere near as good as the Oracle ETEN results. This is not to say that better schemes cannot be devised (and I believe they can), but a moot point if you agree with Craig that there is not much benefit to be realized to begin with even with perfect instantaneous knowledge of losses. Best Regards, Rajesh From swati at seas.upenn.edu Sun Feb 20 17:48:58 2005 From: swati at seas.upenn.edu (Saswati Sarkar) Date: Sun, 20 Feb 2005 20:48:58 -0500 Subject: [e2e] Announcement and Call for papers for SIGCOMM 2005 workshops Message-ID: <1108950538.42193e0a8e50e@webmail.seas.upenn.edu> SIGCOMM Workshops: 2005, Auguat 22-26, Philadelphia ANNOUNCEMENT and CALL FOR PAPERS This year, SIGCOMM 2005 continues its expanded scope with significant emphasis on workshops. We solicit papers and participation for the following one-day workshops, that will be held in conjunction with SIGCOMM 2005 from August 22 - August 26, 2005 in Philadelphia, PA, US. For a detailed description of the workshops and submission guidelines, visit: http://www.acm.org/sigcomm/sigcomm2005/workshops.html 1. Workshop on mining network data (MineNet-05) Todays IP networks are extensively instrumented for collecting a wealth of information including traffic traces (e.g., packet or flow level traces), control (e.g., router forwarding tables, BGP and OSPF updates), and management (e.g., alarms, SNMP traps) data. The real challenge is to process and analyze this vast amount of primarily unstructured information and extract structures, relationships, and higher level knowledge embedded in it and use it to aid network management and operations. The goal of this one day workshop is to explore new directions in network data collection, storage, and analysis techniques, and their application to network monitoring, management, and remediation. The workshop will provide a venue for researchers and practitioners from different backgrounds, including networking, data mining, machine learning, and statistics, to get together and collaboratively approach this problem from their respective vantage points. 2. Workshop on experimental approaches to wireless network design and analysis (E-WIND-05) Research in wireless networking is rapidly becoming more experimental. Research prototypes are being developed for systems ranging from large-scale sensor networks to high-speed wireless access networks. Moreover experimentation and measurement studies are being performed with off-the-shelf hardware and operational testbeds. The goal of this workshop is to bring together experimentalist researchers from diverse backgrounds including wireless hardware platforms, wireless communications, wireless testbeds, and measurement of deployed wireless systems. The workshop will provide a forum for exchange of ideas, challenges, and work-in-progress discussions between both the wireless and wireline measurement communities. The workshop will leave a large space to discussion and possible coordination. 3. Workshop on delay tolerant networking and related networks (WDTN-05) Today, the most successful network architecture is that of the Internet. It has scaled well beyond the original plan of its designers, and the Internet Protocol has been carried on a great number of underlying protocols, including itself. However, the Internet's protocol architecture suffers some problems when implemented on classes of networks for which it was not originally designed. For example, when disconnection and reconnection is common, or link performance is highly variable or extreme, one or more of the traditional Internet protocols do not work well. In this workshop, we wish to explore physical networks that operate significantly differently from wired, connected networks and the protocol architectures and algorithms used to deal with such situations. Techniques for making applications tolerant to disruptions and/or high delays are also requested. 4. Workshop on economics of peer-to-peer systems (P2PECON-05) From file-sharing to distributed computation, from application layer overlays to mobile ad hoc networking, the ultimate success of a peer-to-peer system rests on the twin pillars of scalable and robust system design and alignment of economic interests among the participating peers. Following the success of the first two workshops, the Third Workshop on Economics of Peer-to-Peer Systems will again bring together researchers and practitioners from multiple disciplines to discuss the economic characteristics of P2P systems, application of economic theories to P2P system design, and future directions and challenges in this area. ----- End forwarded message ----- Saswati Sarkar Assistant Professor Department of Electrical and Systems Engineering University of Pennsylvania Email: swati at seas.upenn.edu Phone: 2155739071 Fax: 2155732068 Webpage: http://www.seas.upenn.edu/~swati Mail: 354 Moore, 200 S. 33rd street Philadelphia, PA 19107 USA From craig at aland.bbn.com Wed Feb 23 09:14:57 2005 From: craig at aland.bbn.com (Craig Partridge) Date: Wed, 23 Feb 2005 12:14:57 -0500 Subject: [e2e] Routers accessing TCP header In-Reply-To: Your message of "Wed, 23 Feb 2005 08:53:53 GMT." <1ef22590050223005339b9c079@mail.gmail.com> Message-ID: <20050223171457.4655A243@aland.bbn.com> Hi Arjuna: Thanks for the note placing your work in context. I downloaded a couple of the tech reports and read them briefly -- it helps to understand that you came at the problem of error notification as a way to help avoid the unnecessary retransmission due to dupe acks problem. Let me explain the original point of my note. Bob Braden (manager of this list) is often fond of reminding folks that we, in computer science, do a poor job of seeking out other's work before we start on our own. So a lot of work starts with a blank piece of paper, when a lot of prior work would ensure the work started better informed. So I encourage you to look not so much at our ETEN paper's contents (which is addressing a different issue) but at our references (some 40+) which seek to cite all the prior work worrying about error notification -- you may find something relevant to your work. Good luck! Craig From arjuna.sathiaseelan at gmail.com Wed Feb 23 09:48:37 2005 From: arjuna.sathiaseelan at gmail.com (Arjuna Sathiaseelan) Date: Wed, 23 Feb 2005 17:48:37 +0000 Subject: [e2e] Routers accessing TCP header In-Reply-To: <20050223171457.4655A243@aland.bbn.com> References: <1ef22590050223005339b9c079@mail.gmail.com> <20050223171457.4655A243@aland.bbn.com> Message-ID: <1ef22590050223094874229c0c@mail.gmail.com> Dear Craig, Thanks for the reply. Ooops, my tech reports on the web are like child's play :)..The experiments performed and lots of issues were done during my early stages and are void..I will be updating them with my proper work soon and let you know. Once again, thanks for your advice. Regards, Arjuna On Wed, 23 Feb 2005 12:14:57 -0500, Craig Partridge wrote: > > Hi Arjuna: > > Thanks for the note placing your work in context. I downloaded a couple of > the tech reports and read them briefly -- it helps to understand that you > came at the problem of error notification as a way to help avoid the > unnecessary retransmission due to dupe acks problem. > > Let me explain the original point of my note. Bob Braden (manager of > this list) is often fond of reminding folks that we, in computer science, > do a poor job of seeking out other's work before we start on our own. > So a lot of work starts with a blank piece of paper, when a lot of prior > work would ensure the work started better informed. So I encourage you > to look not so much at our ETEN paper's contents (which is addressing > a different issue) but at our references (some 40+) which seek to cite > all the prior work worrying about error notification -- you may find > something relevant to your work. > > Good luck! > > Craig > From arjuna.sathiaseelan at gmail.com Wed Feb 23 11:34:46 2005 From: arjuna.sathiaseelan at gmail.com (Arjuna Sathiaseelan) Date: Wed, 23 Feb 2005 19:34:46 +0000 Subject: [e2e] Routers accessing TCP header In-Reply-To: <1ef22590050223094874229c0c@mail.gmail.com> References: <1ef22590050223005339b9c079@mail.gmail.com> <20050223171457.4655A243@aland.bbn.com> <1ef22590050223094874229c0c@mail.gmail.com> Message-ID: <1ef22590050223113436622441@mail.gmail.com> Dear Craig, I have updated the files and you can access them here incase you are interested: "Reorder Notifying TCP with Explicit Packet Drop Notification" http://www.dcs.kcl.ac.uk/pg/arjuna/rn-tcp.pdf "Robust TCP with Explicit Packet Drop Notification for Satelliet Networks" http://www.dcs.kcl.ac.uk/pg/arjuna/tcp-r.pdf Regards, Arjuna On Wed, 23 Feb 2005 17:48:37 +0000, Arjuna Sathiaseelan wrote: > Dear Craig, > Thanks for the reply. Ooops, my tech reports on the web are like > child's play :)..The experiments performed and lots of issues were > done during my early stages and are void..I will be updating them with > my proper work soon and let you know. Once again, thanks for your > advice. > > Regards, > Arjuna > > > On Wed, 23 Feb 2005 12:14:57 -0500, Craig Partridge wrote: > > > > Hi Arjuna: > > > > Thanks for the note placing your work in context. I downloaded a couple of > > the tech reports and read them briefly -- it helps to understand that you > > came at the problem of error notification as a way to help avoid the > > unnecessary retransmission due to dupe acks problem. > > > > Let me explain the original point of my note. Bob Braden (manager of > > this list) is often fond of reminding folks that we, in computer science, > > do a poor job of seeking out other's work before we start on our own. > > So a lot of work starts with a blank piece of paper, when a lot of prior > > work would ensure the work started better informed. So I encourage you > > to look not so much at our ETEN paper's contents (which is addressing > > a different issue) but at our references (some 40+) which seek to cite > > all the prior work worrying about error notification -- you may find > > something relevant to your work. > > > > Good luck! > > > > Craig > > > From krash at bbn.com Wed Feb 23 13:46:17 2005 From: krash at bbn.com (Rajesh Krishnan) Date: Wed, 23 Feb 2005 16:46:17 -0500 (EST) Subject: [e2e] Routers accessing TCP header In-Reply-To: <0511C607B17F804EBE96FFECD1FD98591E829B@cs-exs2.cs-nt.bu.edu> from "Ibrahim Matta" at Feb 23, 2005 11:37:59 AM Message-ID: <200502232146.j1NLkH605589@a.bbn.com> Ibrahim, When the oracle notifies the TCP sender of a segment lost due to corruption, we have the sender retransmit the segment immediately without adjusting the congestion window. (Drops due to congestion are discovered and handled using the normal TCP mechanisms.) In our study, the sender did not have explicit information about other flows. In addition to the Oracle case, we tried a couple of cumulative notification strategies that require (intermediate routers attached to error-prone links to update) a header field that accumulates a corruption survivability metric along the path. Upon a loss event, based on the metric the sender can either - probabilistically decide the loss is due to corruption and retransmit without reducing the congestion window - deterministically compute a multiplicative decrease factor that depends on the corruption survivability metric rahter than the tradition value TCP uses (one half) Best Regards, Rajesh > I admit not reading this paper (yet:) But I am wondering what kind of > recovery strategy is used (e.g. TCP not backing off completely, or...) > once you know the "exact" reason of loss (through your oracle). I think > how much gain will depend not only on the "binary" classification of > loss type (e.g. buffer overflow or transmission error) but also on the > action TCP flows take, how synchronized they are, etc. Actually, a finer > grained error classification (e.g. short-term vs. long-term error > conditions etc.) along with a finer grained recovery response maybe > needed? > > Best, > Ibrahim > > > -----Original Message----- > From: end2end-interest-bounces at postel.org > [mailto:end2end-interest-bounces at postel.org] On Behalf Of Rajesh > Krishnan > Sent: Tuesday, February 22, 2005 2:55 PM > To: Craig Partridge > Cc: end2end-interest at postel.org; Arjuna Sathiaseelan > Subject: Re: [e2e] Routers accessing TCP header > > > > >Dear Jon, > > > Thanks for your appropriate reply. I am basically working on a > > >mechanism called the Explicit Packet Drop Notification (EPDN). > > > > While the mechanism is slightly different, this sounds like Explicit > > Transport Error Notification (ETEN) done again. See: > > > > Rajesh Krishnan, James P.G. Sterbenz, Wesley M. Eddy, Craig > Partridge > > and Mark Allman, "Explicit transport error notification (ETEN) for > > > error-prone wireless and satellite networks," Computer Networks, > > 46(3), October 2004, pp. 343-362. > > > > I'll note that my co-authors and I continue to debate exactly how much > > > benefit such a service provides. My reaction, from the "Oracle ETEN" > > simulations in the paper (where we report the maximum improvement > > possible if you know perfectly and immediately which packets are lost) > > > is that the gains are not enough to merit implementing the service. > > Your mileage may vary! > > > > Craig > > The Oracle ETEN plots do show that there are benefits from notifications > > at _high_ error rates. The debate is about: > > - are the high error rates at which we see appreciable gains from > notifications just too disruptive to be usable in practice (e.g. > even for neighbor discovery and routing protocols to function > properly, let alone run TCP) > > - are the environments that can benefit from notifications just too > special to merit retrofitting a general-purpose transport protocol > (and would it not be more efficient to handle these special > anomalies > locally rather than end-to-end) > > Notifications on a per-packet basis consume additional network resources > > and are potentially a security risk as well; schemes that can work with > cumulative notifications are desirable. Unfortunately the performance > of the cumulative scheme we developed was nowhere near as good as the > Oracle ETEN results. This is not to say that better schemes cannot be > devised (and I believe they can), but a moot point if you agree with > Craig that there is not much benefit to be realized to begin with even > with perfect instantaneous knowledge of losses. > > Best Regards, > Rajesh > From tvest at pch.net Wed Feb 23 14:45:54 2005 From: tvest at pch.net (Tom Vest) Date: Thu, 24 Feb 2005 07:45:54 +0900 Subject: [e2e] Measuring the benefits of statistical multiplexing Message-ID: Would be grateful if someone could point me to books/papers that attempt to empirically measure the benefits of statistical multiplexing.... Thanks in advance, Tom From s.malik at tuhh.de Fri Feb 25 05:25:55 2005 From: s.malik at tuhh.de (Sireen Habib Malik) Date: Fri, 25 Feb 2005 14:25:55 +0100 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: <421D6A2F.2A5C2496@attglobal.net> References: <000a01c50f95$98bfd660$f01aeb80@snow> <20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> <20050218181852.GO90698@pun.isi.edu> <4216F217.647A24AC@attglobal.net> <20050221180702.GD16091@pun.isi.edu> <421AA784.1BB0A1CC@attglobal.net> <421B0946.1000406@tuhh.de> <421D6A2F.2A5C2496@attglobal.net> Message-ID: <421F2763.4020305@tuhh.de> Hi, First thanks to Mr. Cannara's who sent me very valuable research papers in response to my last email. Thanks also for zipping them. For a person who decided to improve his understanding of TCP only a few days ago, i do feel that the sent papers cover a remarkable breadth of the top TCP research. Anyone also walking to same line, please send me an email, i will forward the papers. I come to the point Mr. Cannara raised in his email(s): losses. If the key word is performance then my feeling is that more than losses, TCP's bottleneck is its dependence on RTT. Secondly, its tendency to send segments in correlated manner causing burstiness in the small time scales leading to higher queue build-up. I think the important point to approach TCP modelling is not to straight away go to the calculation of losses but first understand its key property: bandwidth sharing. For example, we have N flows over C capacity link with B buffer. The flows are "not" snychronized and are saturating the link. Then for each connection, average_rate = C/N. Immediately one can establish the average_window. average_window/RTT = C/N If i want to make it even more precise. average_window/(RTT+2B/C) = C/N, or average_window= (RTT*C + 2B)/N, as each connection will see full buffer before loosing a packet, B/C is the time to drain the buffer. I have added two buffer delays assuming symmetric bidirectional traffic. Actually it should be RTT+kB/C, where k indicates the traffic assymetry. This is quite intuitive as the total pipe volume and the buffer space will be shared by N flows. For constant RTT+kB, we have the average_window inversely proportional to the number of connections. A linear relationship! This is very nice. If not for its dependence on RTT, the protocol looks very FAIR! Now TP= avg_window/(RTT+2B/C). Therefore, dependence on RTT not only chokes the TP but also makes it unfair! Losses then seem to me as a second order issue, something that TCP has to do to adjust its average window. I suppose with active queue management one could take care of that. For the point about correlated packets, the solution looks like putting negative exponentially distributed time-gaps between the departing packets. Just a rough idea. Anyways, this is what i think could be fundamentally improved in the protocol. Please do feel expand our understanding, this is after all a discussion group. regards, SM From lxu2 at unity.ncsu.edu Fri Feb 25 07:04:51 2005 From: lxu2 at unity.ncsu.edu (Lisong Xu) Date: Fri, 25 Feb 2005 09:04:51 -0600 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: <421F2763.4020305@tuhh.de> References: <000a01c50f95$98bfd660$f01aeb80@snow> <20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> <20050218181852.GO90698@pun.isi.edu> <4216F217.647A24AC@attglobal.net> <20050221180702.GD16091@pun.isi.edu> <421AA784.1BB0A1CC@attglobal.net> <421B0946.1000406@tuhh.de> <421D6A2F.2A5C2496@attglobal.net> <421F2763.4020305@tuhh.de> Message-ID: <421F3E93.6040406@unity.ncsu.edu> Here is a related paper by RPI, which introduces uniformly distributed delays to TCP packets. http://www.ecse.rpi.edu/Homepages/shivkuma/research/papers/randomized-infocom.pdf Regards Lisong Sireen Habib Malik wrote: > Hi, > > First thanks to Mr. Cannara's who sent me very valuable research papers > in response to my last email. Thanks also for zipping them. > For a person who decided to improve his understanding of TCP only a few > days ago, i do feel that the sent papers cover a remarkable breadth of > the top TCP research. Anyone also walking to same line, please send me > an email, i will forward the papers. > I come to the point Mr. Cannara raised in his email(s): losses. > > If the key word is performance then my feeling is that more than losses, > TCP's bottleneck is its dependence on RTT. Secondly, its tendency to > send segments in correlated manner causing burstiness in the small time > scales leading to higher queue build-up. > > I think the important point to approach TCP modelling is not to straight > away go to the calculation of losses but first understand its key > property: bandwidth sharing. > For example, we have N flows over C capacity link with B buffer. The > flows are "not" snychronized and are saturating the link. > Then for each connection, > > average_rate = C/N. > > Immediately one can establish the average_window. > > average_window/RTT = C/N > > If i want to make it even more precise. > > average_window/(RTT+2B/C) = C/N, > > or average_window= (RTT*C + 2B)/N, > > as each connection will see full buffer before loosing a packet, B/C is > the time to drain the buffer. I have added two buffer delays assuming > symmetric bidirectional traffic. Actually it should be RTT+kB/C, where k > indicates the traffic assymetry. > This is quite intuitive as the total pipe volume and the buffer space > will be shared by N flows. > > For constant RTT+kB, we have the average_window inversely proportional > to the number of connections. A linear relationship! This is very nice. > If not for its dependence on RTT, the protocol looks very FAIR! > > Now TP= avg_window/(RTT+2B/C). Therefore, dependence on RTT not only > chokes the TP but also makes it unfair! > > Losses then seem to me as a second order issue, something that TCP has > to do to adjust its average window. I suppose with active queue > management one could take care of that. > > For the point about correlated packets, the solution looks like putting > negative exponentially distributed time-gaps between the departing > packets. Just a rough idea. > > Anyways, this is what i think could be fundamentally improved in the > protocol. Please do feel expand our understanding, this is after all a > discussion group. > > > regards, > SM > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From dpreed at reed.com Fri Feb 25 07:06:12 2005 From: dpreed at reed.com (David P. Reed) Date: Fri, 25 Feb 2005 10:06:12 -0500 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: <421F2763.4020305@tuhh.de> References: <000a01c50f95$98bfd660$f01aeb80@snow> <20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> <20050218181852.GO90698@pun.isi.edu> <4216F217.647A24AC@attglobal.net> <20050221180702.GD16091@pun.isi.edu> <421AA784.1BB0A1CC@attglobal.net> <421B0946.1000406@tuhh.de> <421D6A2F.2A5C2496@attglobal.net> <421F2763.4020305@tuhh.de> Message-ID: <421F3EE4.9040503@reed.com> Just two quick high-level observations, since framing the issue properly is critical: 1) TCP is a protocol that is designed for networks, not links. That means that many diverse users with diverse and unpredictable (not gaussian, not time-invariant, not stationary, not independent - i.e. theoretical simplification is nice, but optimizing a theoretical model is only relevant to the limited extent that the real world is like the model.) 2) TCP is designed to be robust over a wide variety of network implementation choices, and should not depend on assumptions like "routes are stable and unchanging". 3) TCP, and the Internet itself, are not designed to maximize the utilization of the wires. The mathematicians looking for objective functions to optimize, and the elders who remember when wires were expensive, but haven't paid attention for 50 years, tend to focus on this objective function. They remind me of people who would decide that a highway system was optimally operating when cars are going 1 mph with 1 inch between them in all directions. Yes, the wires would be full, and the throughput (in cars per minute) would be pretty high, but the latency would suck. A reasonable objective function of the network is the probability that a user walking up to the network to send a message or get an MP3 file or a web page is getting a service that costs less than what the value of the service is to him/her. This has precious little to do with utilization of the wires, it turns out, in any practical network. I'd fire any graduate student or post-doc who thought that utilization was "obviously" the best metric for network performance. Peer reviewers in network protocol performance work who rate purely mathematical exercises highly because they solve irrelevant problems have accepted far too many papers that focus on "utilization" as a metric. Read Andrew Odlyzko's many papers to get a better idea of what metrics actually matter in practice - and understand why 10% peak utilization is a typical, desirable corporate network operating point, and 50% is a sign of bad management. From s.malik at tuhh.de Fri Feb 25 09:11:08 2005 From: s.malik at tuhh.de (Sireen Habib Malik) Date: Fri, 25 Feb 2005 18:11:08 +0100 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: <421F3EE4.9040503@reed.com> References: <000a01c50f95$98bfd660$f01aeb80@snow> <20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> <20050218181852.GO90698@pun.isi.edu> <4216F217.647A24AC@attglobal.net> <20050221180702.GD16091@pun.isi.edu> <421AA784.1BB0A1CC@attglobal.net> <421B0946.1000406@tuhh.de> <421D6A2F.2A5C2496@attglobal.net> <421F2763.4020305@tuhh.de> <421F3EE4.9040503@reed.com> Message-ID: <421F5C2C.3000507@tuhh.de> Lisong, thanks for the paper. Very nice to know that out of the two things that i raised in my previous email is already taken care off :-) Dear Mr. Reed, Honestly, i will not pretend that i have fully grasped the depth of your message. So please bear with me. Internet pipes are < 15-20% utilized and 90% of it is TCP traffic. You are also right that utilization is not the best metric to look at when the links are so low occupied. It also makes sense to keep them below 50%: all queuing models tell us that, plus capacity for protection/restoration, and if i recall correctly from one of Mr. Odlyzko's more recent papers the Internet traffic grows still 50% per year, so networks should have extra Mbps for the future. Agreed, TCP is designed for the whole network. However, as i undestand it, if the Maximum Congestion Window allows the TCP to discover bandwidth, at one point it will hit a bottleneck. It is a bottleneck because it is saturated! That "link" will then ulitmately determine its performance. Am i right? If not then I do not understand why should not TCP discover and "share" all the available bandwidth that it "can"? I have put "can" in quotes as it co-operates with other TCPs to share the bandwidth. I do not understand the point about optimization. I agree that one very common objective function is to "minimize" the maximum and/or average utilization of the links in the network. I have never seen any paper that says that "maximizing" the utilization is an objective function. Smarter (not the smartest) optimization models bring delays as constraint. Does this not help the losses (Erlang Loss Formula) and delays (M/M/1, M/D/1, MG1-Processor Sharing, Semi-Markov/M/1, etc.), and achieve exactly the same objective of separating the cars by more than one inch? I agree with your take on the assumptions of stationarity and independence: self-similar behaviour extends from scales greater than RTT to minutes, hours, and perhaps days and months. If i put your comment in the context of three variables that i discussed in my last email, then Capacity and Buffer Size are constant, however, Number of Flows (N) is a random variable. Even if it is heavy-tailed variable, it would be fair to assume that the first moment, or the average exists. So the analysis still has relevancy if assume N is the average number of active flows. If its a non-stationary process then atleast the assumption of steady behaviour for a few seconds, or minutes, will hold the analysis true for those scales. regards, Sireen Malik David P. Reed wrote: > Just two quick high-level observations, since framing the issue > properly is critical: > > 1) TCP is a protocol that is designed for networks, not links. That > means that many diverse users with diverse and unpredictable (not > gaussian, not time-invariant, not stationary, not independent - i.e. > theoretical simplification is nice, but optimizing a theoretical model > is only relevant to the limited extent that the real world is like the > model.) > > 2) TCP is designed to be robust over a wide variety of network > implementation choices, and should not depend on assumptions like > "routes are stable and unchanging". > > 3) TCP, and the Internet itself, are not designed to maximize the > utilization of the wires. The mathematicians looking for objective > functions to optimize, and the elders who remember when wires were > expensive, but haven't paid attention for 50 years, tend to focus on > this objective function. They remind me of people who would decide > that a highway system was optimally operating when cars are going 1 > mph with 1 inch between them in all directions. Yes, the wires would > be full, and the throughput (in cars per minute) would be pretty high, > but the latency would suck. > > A reasonable objective function of the network is the probability that > a user walking up to the network to send a message or get an MP3 file > or a web page is getting a service that costs less than what the value > of the service is to him/her. This has precious little to do with > utilization of the wires, it turns out, in any practical network. > I'd fire any graduate student or post-doc who thought that utilization > was "obviously" the best metric for network performance. > > Peer reviewers in network protocol performance work who rate purely > mathematical exercises highly because they solve irrelevant problems > have accepted far too many papers that focus on "utilization" as a > metric. Read Andrew Odlyzko's many papers to get a better idea of > what metrics actually matter in practice - and understand why 10% peak > utilization is a typical, desirable corporate network operating point, > and 50% is a sign of bad management. > From faber at ISI.EDU Fri Feb 25 11:41:29 2005 From: faber at ISI.EDU (Ted Faber) Date: Fri, 25 Feb 2005 11:41:29 -0800 Subject: [e2e] link between Kelly's control and TCP's AIMD In-Reply-To: <421F2763.4020305@tuhh.de> References: <20050217192035.GA77148@pun.isi.edu> <4215795A.CC886DFD@attglobal.net> <20050218181852.GO90698@pun.isi.edu> <4216F217.647A24AC@attglobal.net> <20050221180702.GD16091@pun.isi.edu> <421AA784.1BB0A1CC@attglobal.net> <421B0946.1000406@tuhh.de> <421D6A2F.2A5C2496@attglobal.net> <421F2763.4020305@tuhh.de> Message-ID: <20050225194129.GG62493@pun.isi.edu> On Fri, Feb 25, 2005 at 02:25:55PM +0100, Sireen Habib Malik wrote: > Hi, > > First thanks to Mr. Cannara's who sent me very valuable research papers in > response to my last email. Thanks also for zipping them. I'm curious. Please send citations to the papers to e2e. Do not send me a zip file. Citations are quite sufficient. -- Ted Faber http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.asc Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://www.postel.org/pipermail/end2end-interest/attachments/20050225/2d5a5b21/attachment.bin From kx2 at njit.edu Fri Feb 25 12:19:39 2005 From: kx2 at njit.edu (Roy Xu) Date: Fri, 25 Feb 2005 15:19:39 -0500 Subject: [e2e] 10% packet loss stops TCP flow Message-ID: <001901c51b77$55a51eb0$0a1aeb80@snow> Hi all, it seems to be a common understanding that if a TCP flow experiences 10% or more packet loss, the flow stops (i.e., attains 0 or meaningless throughput) ns-2 simulations also seem to agree with this observation. my questions is what is the theoretical or analytical explanation for this observation? thanks in advance. --roy -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.postel.org/pipermail/end2end-interest/attachments/20050225/b150fd39/attachment.html From biazsaa at eng.auburn.edu Fri Feb 25 12:30:55 2005 From: biazsaa at eng.auburn.edu (Saad Biaz) Date: Fri, 25 Feb 2005 14:30:55 -0600 (CST) Subject: [e2e] 10% packet loss stops TCP flow In-Reply-To: <001901c51b77$55a51eb0$0a1aeb80@snow> Message-ID: You can either find a simplistic model in a paper by Mathis et al. "The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm" or look at the model developed by Padhye et al. "Modeling TCP Throughput: A Simple Model and its Empirical.." Hope this helps" On Fri, 25 Feb 2005, Roy Xu wrote: > Hi all, > > it seems to be a common understanding that if a TCP flow experiences > 10% or more packet loss, the flow stops (i.e., attains 0 or meaningless throughput) > > ns-2 simulations also seem to agree with this observation. > > my questions is what is the theoretical or analytical explanation for this observation? > > thanks in advance. > --roy > ---------------------------------------------------------------------- Saad Biaz, Ph.D., Assistant Prof. Voice: (334) 844 6307 114 Dunstan Hall Auburn University Fax : (334) 844 6329 Auburn, AL 36849-5347, USA http://www.eng.auburn.edu/~biazsaa ---------------------------------------------------------------------- From jnc at mercury.lcs.mit.edu Fri Feb 25 12:40:22 2005 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Fri, 25 Feb 2005 15:40:22 -0500 (EST) Subject: [e2e] 10% packet loss stops TCP flow Message-ID: <20050225204022.68BB386AE1@mercury.lcs.mit.edu> > From: Roy Xu > it seems to be a common understanding that if a TCP flow experiences > 10% or more packet loss, the flow stops (i.e., attains 0 or meaningless > throughput) > ... > my questions is what is the theoretical or analytical explanation for > this observation? Exercise for the reader: If a TCP connection will time out and abort the connection if it receives no ACK to T retransmissions of a segment X, given that it has N total segments to transmit, derive the equation f which gives Pc = f(Pp, T, N), where Pc is the probability of the successful completion of the tranmssion, and Pp is the probability of the loss of any individual packet. Noel From craig at aland.bbn.com Fri Feb 25 13:11:26 2005 From: craig at aland.bbn.com (Craig Partridge) Date: Fri, 25 Feb 2005 16:11:26 -0500 Subject: [e2e] 10% packet loss stops TCP flow In-Reply-To: Your message of "Fri, 25 Feb 2005 15:19:39 EST." <001901c51b77$55a51eb0$0a1aeb80@snow> Message-ID: <20050225211126.3E2101FF@aland.bbn.com> 10% sounds a little low for the connection to regularly fail (poor throughput yes, failure no). But a few points. 10% loss means that about once every 10 segments sent, you'll reset the congestion window and restart slow start/congestion avoidance. In general, you'll find that ssthresh gets stuck around 2 to 3, and so you're running at or close to 1 or 2 segments outstanding at time. That's horrible throughput but functional. The next step is to figure out when the connection dies. If you're in one segment at a time mode, that means that for a round-trip to succeed, your data segment needs to get through and an ack needs to come back. (Acks don't come back spontaneously, only in response to retransmissions, and since the window size is 1, you don't get the win of cumulative acks). So the chance of a successful round-trip is .9*.9 or .81 -- failure is .19 TCP's typically retransmit 7 to 10 times (if I'm remembering right). The probability of 7 straight failures is (.19)^7 or around 10^-5. So a connection of about 10,000 segments (with large variance around that number) just plain fails. Of course, this assumes loss is independent. If losses tend to cluster, you'll launch your retransmissions straight into the loss episode and lose much faster. There are probably much better models, but these work pretty well for rough envelope calculations in my experience. Craig From jonathan at dsg.stanford.edu Fri Feb 25 13:58:25 2005 From: jonathan at dsg.stanford.edu (Jonathan Stone) Date: Fri, 25 Feb 2005 13:58:25 -0800 Subject: [e2e] 10% packet loss stops TCP flow In-Reply-To: Your message of "Fri, 25 Feb 2005 15:40:22 EST." <20050225204022.68BB386AE1@mercury.lcs.mit.edu> Message-ID: In message <20050225204022.68BB386AE1 at mercury.lcs.mit.edu>, Noel Chiappa writes: > > From: Roy Xu > > > it seems to be a common understanding that if a TCP flow experiences > > 10% or more packet loss, the flow stops (i.e., attains 0 or meaningless > > throughput) > > ... > > my questions is what is the theoretical or analytical explanation for > > this observation? > >Exercise for the reader: If a TCP connection will time out and abort the >connection if it receives no ACK to T retransmissions of a segment X, given >that it has N total segments to transmit, derive the equation f which gives >Pc = f(Pp, T, N), where Pc is the probability of the successful completion of >the tranmssion, and Pp is the probability of the loss of any individual >packet. Noel, I'm puzzled. Are you deliberately tempting your readers to fall into the fallacy of assuming packet loss is statistically independent, and thus of assuming Poisson behaviour of packets in the Internet? If so, sure, it's an educational exercise who haven't gotten that far. But even so... for shame, sir :-/. From jnc at mercury.lcs.mit.edu Fri Feb 25 15:08:10 2005 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Fri, 25 Feb 2005 18:08:10 -0500 (EST) Subject: [e2e] 10% packet loss stops TCP flow Message-ID: <20050225230810.1244E86AE5@mercury.lcs.mit.edu> > From: Jonathan Stone >> T retransmissions .. given that it has N total segments to transmit, >> .. Pc = f(Pp, T, N), where Pc is the probability of the successful >> completion of the tranmssion, and Pp is the probability of the loss of >> any individual packet. > Are you deliberately tempting your readers to fall into the fallacy of > assuming packet loss is statistically independent, and thus of assuming > Poisson behaviour of packets in the Internet? Well, it depends on what's causing the packet loss, doesn't it? E.g. if it's congestion, yes there is a chance it will be non-random (although it will depend on what drop algorithm the switches/routers are using). However, if it's a lossy radio link, it might be fairly well random. May I also note that Pc calculated by the simplistic formula I point toward is actually something of an *upper* bound, and any deviation away from perfect randomness in packet dropping will *reduce* it. (If the losses are not evenly randomly distributed, then the probability of the loss of the T retransmission of a given packet needed to kill the connection is even *higher* than the Pp^T you get with a random model, no?) Noel From jonathan at dsg.stanford.edu Fri Feb 25 16:08:04 2005 From: jonathan at dsg.stanford.edu (Jonathan Stone) Date: Fri, 25 Feb 2005 16:08:04 -0800 Subject: [e2e] 10% packet loss stops TCP flow In-Reply-To: Your message of "Fri, 25 Feb 2005 18:08:10 EST." <20050225230810.1244E86AE5@mercury.lcs.mit.edu> Message-ID: In message <20050225230810.1244E86AE5 at mercury.lcs.mit.edu>, Noel Chiappa writes: > > From: Jonathan Stone > > Are you deliberately tempting your readers to fall into the fallacy of > > assuming packet loss is statistically independent, and thus of assuming > > Poisson behaviour of packets in the Internet? > >Well, it depends on what's causing the packet loss, doesn't it? Yes, very much so. And calculating a back-of-the-envelope approximation (like Craig's) is clearly much better than asking here. >E.g. if it's >congestion, yes there is a chance it will be non-random (although it will >depend on what drop algorithm the switches/routers are using). However, if >it's a lossy radio link, it might be fairly well random. Maybe, maybe not. I used to use RF loss with Metricom radios as a very similar example. But Stuart Cheshire was keen to remind me that in the environments we saw loss (think lecture halls full of students with laptops, with growing fraction of wireless users), those lossy RF links may well be lossy due to congestion. And (at least with Metricoms), not all radios are equal. Nor all lecture halls. I've recently been debugging a TCP SACK implmeentation, involving several hours staring at tcptrace graphs from my home LAN. I have two 802.11 cards, one of which shows significantly higher drops than the other. wifi-to-wifi drop looks persistently bursty, and the rate of "hardware duplicates" (same IP-id, same TCP segment size and sequence number) is over 1%, which I find disturbingly high. I have no idea how representative that is. >May I also note that Pc calculated by the simplistic formula I point toward is >actually something of an *upper* bound, and any deviation away from perfect >randomness in packet dropping will *reduce* it. (If the losses are not evenly >randomly distributed, then the probability of the loss of the T retransmission >of a given packet needed to kill the connection is even *higher* than the Pp^T >you get with a random model, no?) I think it depends. Reduce in aggregate, when summed over all users? I'd buy that. But there could well be some poor soul who consistently loses to other users, due to RF/building geometry, or (FM) capture effect, or whatever other reasons apply. And those poor souls can be consistently worse off than with statistically-independent loss --- Rather like my former coax neighbour whose Ethernet card persistently garbled outbound packets. If backbone utilization is really on the order of 10%, I wonder what fraction of aggregate end-to-end drop is due to domestic or office wireless congestion, or ether-over-cable-TV adoption rates (some might call it oversubscription); and what those drop distributions look like. From lshlv at xidian.edu.cn Fri Feb 25 20:09:11 2005 From: lshlv at xidian.edu.cn (bryansheng) Date: Sat, 26 Feb 2005 12:09:11 +0800 Subject: [e2e] the implementation of REM algorithm Message-ID: <200502260354.j1Q3s5N9000349@ns2.xidian.edu.cn> Hi, REM algorithm includes source algorithm and link algorithm.Its implementation in AQM need the cooperation of source algorithm.But tcp algorithm at present is different from the source algorithm,so REM cannot be implemented in current internet.To my great surprise,many papers compare its performance with RED and other AQM Algorithms using ns.Who tell me why?Thanks! 11:59:57,2005-02-26 lshlv at xidian.edu.cn best regards, bryansheng From demeer at fmi.uni-passau.de Sat Feb 26 02:15:08 2005 From: demeer at fmi.uni-passau.de (H.DeMeer) Date: Sat, 26 Feb 2005 11:15:08 +0100 Subject: [e2e] REMINDER: Call for short papers 13th EuroNGI / IEEE / IFIP / GI IWQoS 2005 Message-ID: <6.0.1.1.0.20050226110248.035ff328@yoda.fmi.uni-passau.de> >REMINDER: Call for short papers IWQoS'05 > > > > Thirteenth EuroNGI / IEEE / IFIP / GI International Workshop on Quality > of Service (IWQoS 2005) > June 21-23, 2005 > University of Passau, Germany > > > > >2x Short Paper Sessions: "Work in Progress: Innovative, Provocative and Visionary Statements" >"The Impact of QoS: Where Industry meets Academia", see: > >http://www.fmi.uni-passau.de/lehrstuehle/demeer/iwqos/short_paper.html > >These sessions invite short paper submissions, up to 5-6 double-spaced >single-column >pages with font sizes of 11 or larger, including all figures and references. > >Accepted papers are limited to three single-spaced pages, >will be presented in a 15-minute time period, and will be included in the >IWQoS'05 conference proceedings. > >Important Dates: > * Short paper submission deadline: March 7th, 2005, 11:59pm CET > * Notification of acceptance: March 24th , 2005 > * Camera-ready papers due: April 8, 2005 > * Hotel reservation cut-off date: May 19th , 2005 > * Early registration deadline: April 8th, 2005 > * Workshop dates: June 21-23 , 2005 > * Workshop reception and welcome party: June 20th, 2005 >Session chairs: > >Francois Le Faucheur, Cicso Systems, France > >Georgios Karagiannis, University of Twente, The Netherlands Program chairs: Nina Bhatti, HP Labs, Palo Alto, USA Hermann de Meer, University of Passau, Germany >_______________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.postel.org/pipermail/end2end-interest/attachments/20050226/6e318aaf/attachment-0001.html From Jon.Crowcroft at cl.cam.ac.uk Sat Feb 26 02:26:39 2005 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Sat, 26 Feb 2005 10:26:39 +0000 Subject: [e2e] 10% packet loss stops TCP flow In-Reply-To: Message from Roy Xu of "Fri, 25 Feb 2005 15:19:39 EST." <001901c51b77$55a51eb0$0a1aeb80@snow> Message-ID: naive steady state theory: i) whats the link capacity - lets define capacity as the sustainable window, w and without loss of generality, define RTT as 1. ii) so we can workout the possibility of a timeout during a window - its basically enough packets must be lost that fast recovery doesnt work - e.g. 3 possibility of 3 packets out of w are lost - prob. of no packets being lost out of w is (1-p)^w prob. of exactly 1 packets being lost out of w is p*(1-p)^(w-1) prob. of exactly 2 packets being lost out of w is p^2*(1-p)^(w-2) prob. of exactly 3 packets being lost out of w is p^2*(1-p)^(w-2) prob. of _at least 3_ packets being lost - sum the series...leave as excercise.... naive incremental theory: i) slow start sends 1 packet, gets 1 ack, sends 2 packet, gets 2 acks, sends 4 packet, gets 4 acks ... ii) at this point, we have sent 14 packets - chances are something got lost if the loss rate is 10% so with a loss rate greater than a _very small %age_, we never get out of slow start QED. practice - 1/ congestive losses at 10% would be basically unsustainable...i havnt seen that level since >15 years ago except where ome link failures had forced a lot of traffic onto odd routes... 2/ for interference/noise based loss, you'd haev to look at wireless, and wide area, not 802.11: some wide area wireless links might have a loss rate mean of 10% although that is pretty poor - generally link layer fec would prevent a level that bad....although other problems occur that limit throughput (e.g. delay distribution not having a single mean, or even a finite variance.... so tcp rtt estimation being blown away and causing false timeouts:) so avoiding >a few % packet loss is a recommended link layer design choice if at all possible (inter-planetary links might be a place wher its tricky:) In missive <001901c51b77$55a51eb0$0a1aeb80 at snow>, Roy Xu typed: >>This is a multi-part message in MIME format. >> >>--Boundary_(ID_TjX6gYk/qmMftKBZ5wcP9Q) >>Content-type: text/plain; charset=gb2312 >>Content-transfer-encoding: 7BIT >> >>Hi all, >> >>it seems to be a common understanding that if a TCP flow experiences >>10% or more packet loss, the flow stops (i.e., attains 0 or meaningless throughput) >> >>ns-2 simulations also seem to agree with this observation. >> >>my questions is what is the theoretical or analytical explanation for this observation? >> >>thanks in advance. >>--roy From misra at cs.columbia.edu Sat Feb 26 03:02:51 2005 From: misra at cs.columbia.edu (Vishal Misra) Date: Sat, 26 Feb 2005 06:02:51 -0500 Subject: [e2e] 10% packet loss stops TCP flow In-Reply-To: References: Message-ID: <4220575B.7040902@cs.columbia.edu> There is another very simple way to look at it. The most "optimistic" model of TCP, that only looks at congestion avoidance (AIMD) and ignores slow start, timeouts etc., predicts the following relationship between the average window size (w) and loss probability (p): (w^2)*p = 2 Note that this is _independent_ of link bandwidth and RTT. So with a loss probability of 10%, the average window size is going to be of the order 4. If you factor in other protocol details, the number will be even smaller. With a window size of 4 or below, fast recovery will not work and any loss will lead to a timeout (for the average connection). Hence loss rates of the order 10% simply will not work, irrespective of RTT/link bandwidth. -Vishal -- http://www.cs.columbia.edu/~misra/ From Jon.Crowcroft at cl.cam.ac.uk Sat Feb 26 05:42:37 2005 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Sat, 26 Feb 2005 13:42:37 +0000 Subject: [e2e] 10% packet loss stops TCP flow In-Reply-To: Message from Lloyd Wood of "Sat, 26 Feb 2005 12:47:45 GMT." Message-ID: In missive , Lloyd Wood typed: >>Every lecture like an IETF meeting, with no-one learning anything >>because they're staring at their web browsers... it used to be worse - at the height of the .com madness, and during the 1st bush junior presential "election", you could sit at the back of the room and see people flip between screens showing graphs of the republican vote, the cisco share price, and the traffic growth on their backbone I am absolutely certain that some ISPs overprovisioned because their Ops people were in an IETF meeting and lost track of which graph was which... (there are two (at least) other possible mistaken inferences due to swapped graphs, of course, I leave for others to comment but i doubt very much that election officials in florida were looking at netflow stats, or that enron management were innocently browing the democrat black southern community losing their franchise oh no, not at all) cheers jon p.s. what is the "IETF"? From huitema at windows.microsoft.com Sat Feb 26 12:30:59 2005 From: huitema at windows.microsoft.com (Christian Huitema) Date: Sat, 26 Feb 2005 12:30:59 -0800 Subject: [e2e] 10% packet loss stops TCP flow Message-ID: > There is another very simple way to look at it. The most > "optimistic" model of TCP, that only looks at congestion > avoidance (AIMD) and ignores slow start, timeouts etc., predicts > the following relationship between the average window size (w) > and loss probability (p): > > (w^2)*p = 2 > > Note that this is _independent_ of link bandwidth and RTT. So > with a loss probability of 10%, the average window size is going > to be of the order 4. If you factor in other protocol details, > the number will be even smaller. With a window size of 4 or > below, fast recovery will not work and any loss will lead to a > timeout (for the average connection). Hence loss rates of the > order 10% simply will not work, irrespective of RTT/link bandwidth. The formula (w^2)*p = 2 assumes a connection in "congestion avoidance" mode, which is not actually the case if the loss rate is high. If the loss rate is high, the connection will be very often in "slow start" mode, where the approximation is "w*p = 2". If the loss rate is still higher, we move to "w=1", and the adaptation comes from increasing the RTT. The classic rule of thumb is that a link exhibiting more than 1% packet loss is considered broken. By the way, Noel's challenge points to an interesting point. Long transfers are more likely to break than short transfers. If we want to actually transfer large files over poor quality networks, we cannot rely on TCP alone, but we have to split them in small chunks. Much like what BitTorrent does... -- Christian Huitema From jshen_cad at yahoo.com.cn Sun Feb 27 17:35:35 2005 From: jshen_cad at yahoo.com.cn (Jing Shen) Date: Mon, 28 Feb 2005 09:35:35 +0800 (CST) Subject: [e2e] Papers on router performance In-Reply-To: Message-ID: <20050228013535.58081.qmail@web15409.mail.cnb.yahoo.com> To our measurement, header processing's side effect on router performance is not only to the processing logic but also router architecture. Sometimes vendors just show the best result they got in their test bed, but usually it's not performance it could be achived in production environment. For example, ASIC matrix on Alcatel's 7750 could process packets at line speed, but header processing will hit Cisco 7204's CPU which will degrade its performance. > > You might want to take a look at the first and > second ACM SIGCOMM > > Internet Measurement Workshops. In them you might > find some > > performance measurements on various routers. > > http://www.cisco.com/warp/public/765/tools/quickreference/routerperformance.pdf > Jing Shen _________________________________________________________ Do You Yahoo!? ×¢²áÊÀ½çÒ»Á÷Æ·ÖʵÄÑÅ»¢Ãâ·ÑµçÓÊ http://cn.rd.yahoo.com/mail_cn/tag/1g/*http://cn.mail.yahoo.com/ From randy at psg.com Sun Feb 27 18:55:38 2005 From: randy at psg.com (Randy Bush) Date: Mon, 28 Feb 2005 11:55:38 +0900 Subject: [e2e] 10% packet loss stops TCP flow References: Message-ID: <16930.34858.4677.276894@roam.psg.com> > p.s. what is the "IETF"? the naive initials of the internet vendor task force From s.malik at tuhh.de Mon Feb 28 00:13:56 2005 From: s.malik at tuhh.de (Sireen Habib Malik) Date: Mon, 28 Feb 2005 09:13:56 +0100 Subject: [e2e] 10% packet loss stops TCP flow In-Reply-To: References: Message-ID: <4222D2C4.7060502@tuhh.de> Hi, ......sorry to be trivial but please correct me if i think the expression should be (w^2)*p=2.25. I would have worked out the answer the same way but through the above expression. thanks for the clarification. -- Sireen Malik Christian Huitema wrote: >>There is another very simple way to look at it. The most >>"optimistic" model of TCP, that only looks at congestion >>avoidance (AIMD) and ignores slow start, timeouts etc., predicts >>the following relationship between the average window size (w) >>and loss probability (p): >> >> (w^2)*p = 2 >> >>Note that this is _independent_ of link bandwidth and RTT. So >>with a loss probability of 10%, the average window size is going >>to be of the order 4. If you factor in other protocol details, >>the number will be even smaller. With a window size of 4 or >>below, fast recovery will not work and any loss will lead to a >>timeout (for the average connection). Hence loss rates of the >>order 10% simply will not work, irrespective of RTT/link bandwidth. >> >> > >The formula (w^2)*p = 2 assumes a connection in "congestion avoidance" >mode, which is not actually the case if the loss rate is high. If the >loss rate is high, the connection will be very often in "slow start" >mode, where the approximation is "w*p = 2". If the loss rate is still >higher, we move to "w=1", and the adaptation comes from increasing the >RTT. > >The classic rule of thumb is that a link exhibiting more than 1% packet >loss is considered broken. > >By the way, Noel's challenge points to an interesting point. Long >transfers are more likely to break than short transfers. If we want to >actually transfer large files over poor quality networks, we cannot rely >on TCP alone, but we have to split them in small chunks. Much like what >BitTorrent does... > >-- Christian Huitema > > -- Sireen Malik, M.Sc. PhD. Candidate, Communication Networks Hamburg University of Technology, FSP 4-06 (room 3008) Denickestr. 17 21073 Hamburg, Deutschland Tel: +49 (40) 42-878-3387 Fax: +49 (40) 42-878-2941 E-Mail: s.malik at tuhh.de --Everything should be as simple as possible, but no simpler (Albert Einstein) From arjuna.sathiaseelan at gmail.com Mon Feb 28 00:25:12 2005 From: arjuna.sathiaseelan at gmail.com (Arjuna Sathiaseelan) Date: Mon, 28 Feb 2005 08:25:12 +0000 Subject: [e2e] Routers accessing TCP header In-Reply-To: <200502232146.j1NLkH605589@a.bbn.com> References: <0511C607B17F804EBE96FFECD1FD98591E829B@cs-exs2.cs-nt.bu.edu> <200502232146.j1NLkH605589@a.bbn.com> Message-ID: <1ef225900502280025603d0829@mail.gmail.com> Dear Craig and Rajesh, I was going through your ETEN mechanism and some questions arose. Doesnt Explicit Loss Notification solve the problem of distinguishing corruption and congestion effectively ? So is there any need for creating a new mechanism (this includes my EPDN and your ETEN) to detect the cause of loss? Does ELN have any shortfalls? I would be very much obliged if you could shed some light on this . Regds, Arjuna On Wed, 23 Feb 2005 16:46:17 -0500 (EST), Rajesh Krishnan wrote: > Ibrahim, > > When the oracle notifies the TCP sender of a segment lost due to corruption, > we have the sender retransmit the segment immediately without adjusting the > congestion window. (Drops due to congestion are discovered and handled using > the normal TCP mechanisms.) In our study, the sender did not have explicit > information about other flows. > > In addition to the Oracle case, we tried a couple of cumulative notification > strategies that require (intermediate routers attached to error-prone links > to update) a header field that accumulates a corruption survivability metric > along the path. Upon a loss event, based on the metric the sender can either > > - probabilistically decide the loss is due to corruption and retransmit > without reducing the congestion window > > - deterministically compute a multiplicative decrease factor that depends > on the corruption survivability metric rahter than the tradition value > TCP uses (one half) > > Best Regards, > Rajesh > > > I admit not reading this paper (yet:) But I am wondering what kind of > > recovery strategy is used (e.g. TCP not backing off completely, or...) > > once you know the "exact" reason of loss (through your oracle). I think > > how much gain will depend not only on the "binary" classification of > > loss type (e.g. buffer overflow or transmission error) but also on the > > action TCP flows take, how synchronized they are, etc. Actually, a finer > > grained error classification (e.g. short-term vs. long-term error > > conditions etc.) along with a finer grained recovery response maybe > > needed? > > > > Best, > > Ibrahim > > > > > > -----Original Message----- > > From: end2end-interest-bounces at postel.org > > [mailto:end2end-interest-bounces at postel.org] On Behalf Of Rajesh > > Krishnan > > Sent: Tuesday, February 22, 2005 2:55 PM > > To: Craig Partridge > > Cc: end2end-interest at postel.org; Arjuna Sathiaseelan > > Subject: Re: [e2e] Routers accessing TCP header > > > > > > > >Dear Jon, > > > > Thanks for your appropriate reply. I am basically working on a > > > >mechanism called the Explicit Packet Drop Notification (EPDN). > > > > > > While the mechanism is slightly different, this sounds like Explicit > > > Transport Error Notification (ETEN) done again. See: > > > > > > Rajesh Krishnan, James P.G. Sterbenz, Wesley M. Eddy, Craig > > Partridge > > > and Mark Allman, "Explicit transport error notification (ETEN) for > > > > > error-prone wireless and satellite networks," Computer Networks, > > > 46(3), October 2004, pp. 343-362. > > > > > > I'll note that my co-authors and I continue to debate exactly how much > > > > > benefit such a service provides. My reaction, from the "Oracle ETEN" > > > simulations in the paper (where we report the maximum improvement > > > possible if you know perfectly and immediately which packets are lost) > > > > > is that the gains are not enough to merit implementing the service. > > > Your mileage may vary! > > > > > > Craig > > > > The Oracle ETEN plots do show that there are benefits from notifications > > > > at _high_ error rates. The debate is about: > > > > - are the high error rates at which we see appreciable gains from > > notifications just too disruptive to be usable in practice (e.g. > > even for neighbor discovery and routing protocols to function > > properly, let alone run TCP) > > > > - are the environments that can benefit from notifications just too > > special to merit retrofitting a general-purpose transport protocol > > (and would it not be more efficient to handle these special > > anomalies > > locally rather than end-to-end) > > > > Notifications on a per-packet basis consume additional network resources > > > > and are potentially a security risk as well; schemes that can work with > > cumulative notifications are desirable. Unfortunately the performance > > of the cumulative scheme we developed was nowhere near as good as the > > Oracle ETEN results. This is not to say that better schemes cannot be > > devised (and I believe they can), but a moot point if you agree with > > Craig that there is not much benefit to be realized to begin with even > > with perfect instantaneous knowledge of losses. > > > > Best Regards, > > Rajesh > > > > From s.malik at tuhh.de Mon Feb 28 01:54:09 2005 From: s.malik at tuhh.de (Sireen Habib Malik) Date: Mon, 28 Feb 2005 10:54:09 +0100 Subject: [e2e] TCP losses In-Reply-To: References: Message-ID: <4222EA41.2020201@tuhh.de> Hi, Catching up a couple of days late on the discussion.... but not missing the snow, -7C, and bright sunshine all packed in 48 hours (Hamburg is one of the most beautiful towns on this planet). 1. 10% losses ....enough said on that.........however, a relevant query. How long does TCP take to decide that its connection has gone to the dogs and that the Application must do something about it? RFC1122 (section 4.2.3.5) talks about "atleast" 100seconds. Is this applied practically? 2. TCP seems to have no major problems standing on its own. Very simple theory yields the strength of the protocol. Having said that it is also clear that considering the protocol without considering the under-lying layers is not correct. Considering "congestion" alone as cause of losses is not sufficient - the protocol when zipping through layers underneath it, sees all kinds of failures. Those failures which cause the TCP to have "serious" performance problems. Obviously one way to correct those failures is at their respective layers. The other is to make TCP responsible for the 2*E2E loop for both the data transport and fault-recovery functionality. At this point, i am interested in knowing what breaks TCP outside its own congestion related problems. What are those failures? How frequently they occur? Any idea of duration? I would also appreciate it if you could point out wether the solution to the problem is dealt at the TCP layer or elsewhere. For example, I can think of Link Layer Retransmissions in UMTS which relieve TCP some of its 2*E2E non-congestion related fault recovery. The trade off is delay against no-losses. It would be nicer, if the errors relevant for future large bandwidth-delay product IP over DWDM networks be given priority. Any data, papers, guesses...etc. will be appreciated. Thank you and regards, regards, SM From kostas at cs.sunysb.edu Mon Feb 28 03:02:27 2005 From: kostas at cs.sunysb.edu (Kostas Pentikousis) Date: Mon, 28 Feb 2005 06:02:27 -0500 (EST) Subject: [e2e] 10% packet loss stops TCP flow In-Reply-To: <001901c51b77$55a51eb0$0a1aeb80@snow> References: <001901c51b77$55a51eb0$0a1aeb80@snow> Message-ID: On Fri, 25 Feb 2005, Roy Xu wrote: |it seems to be a common understanding that if a TCP flow experiences |10% or more packet loss, the flow stops (i.e., attains 0 or meaningless throughput) Calculating drop rates from packet loss and using that to perform Bernoulli trials is a poor error model for TCP. First, summarily counting how many packets were sent and how many of them were dropped and calculating a dropped rate (say, 10%) captures very little of what happens in reality. And it's particularly inadequate if you go one more step and use that drop rate in an ns2 simulation study. Second, the TCP retransmission rate (the percentage of packets that are retransmitted) cannot always be equated to packet loss rate. And it is the retransmission rate that really matters if you are interested in high TCP throughput. Nevertheless, a rate-based 10% (drop 1 in 10 packets coming by) does not halt TCP -- at least it didn't in our lab tests: K. Pentikousis and H. Badr, "Error modeling for TCP simulations", Proceedings of EUROSIM 2001, Delft, The Netherlands, June 2001, see esp. Fig. 2 (http://www.cs.stonybrook.edu/~kostas/art/) And yes, that was TCP Tahoe and x-kernel ;) |ns-2 simulations also seem to agree with this observation. Last time I checked, the error models in ns2 do NOT drop pure ACKs. Keep that in mind esp. for one-way TCP simulations. Not sure if FullTCP is any different. Best regards, Kostas __________________________________________________________________ Kostas Pentikousis www.cs.stonybrook.edu/~kostas From s.malik at tuhh.de Mon Feb 28 03:22:44 2005 From: s.malik at tuhh.de (Sireen Habib Malik) Date: Mon, 28 Feb 2005 12:22:44 +0100 Subject: [e2e] 10% packet loss stops TCP flow In-Reply-To: <20050225230810.1244E86AE5@mercury.lcs.mit.edu> References: <20050225230810.1244E86AE5@mercury.lcs.mit.edu> Message-ID: <4222FF04.9050204@tuhh.de> Hi, Assume X is the time the TCP is allowed to wait to recover from the fault (RFC1122, TCP Connection Failures, atleast 100seconds). During this time X, if Pp does not improve then Pc approaches 0 with T going higher. It does give the impression that connection has no chance to recover and finish. However, if in X time ACKS come (basically Pp improved as fault is removed) TCP will recover and finish the transmission "some day"!! Seems then Pc=100%! I think the more interesting case is that what is the probability that TCP will tell the application to kill the connection in time X, if Pp is the loss probability, etc.....? regards, SM Noel Chiappa wrote: > > From: Jonathan Stone > > >> T retransmissions .. given that it has N total segments to transmit, > >> .. Pc = f(Pp, T, N), where Pc is the probability of the successful > >> completion of the tranmssion, and Pp is the probability of the loss of > >> any individual packet. > > > Are you deliberately tempting your readers to fall into the fallacy of > > assuming packet loss is statistically independent, and thus of assuming > > Poisson behaviour of packets in the Internet? > >Well, it depends on what's causing the packet loss, doesn't it? E.g. if it's >congestion, yes there is a chance it will be non-random (although it will >depend on what drop algorithm the switches/routers are using). However, if >it's a lossy radio link, it might be fairly well random. > >May I also note that Pc calculated by the simplistic formula I point toward is >actually something of an *upper* bound, and any deviation away from perfect >randomness in packet dropping will *reduce* it. (If the losses are not evenly >randomly distributed, then the probability of the loss of the T retransmission >of a given packet needed to kill the connection is even *higher* than the Pp^T >you get with a random model, no?) > > Noel > > -- Sireen Malik, M.Sc. PhD. Candidate, Communication Networks Hamburg University of Technology, FSP 4-06 (room 3008) Denickestr. 17 21073 Hamburg, Deutschland Tel: +49 (40) 42-878-3387 Fax: +49 (40) 42-878-2941 E-Mail: s.malik at tuhh.de --Everything should be as simple as possible, but no simpler (Albert Einstein) From s.malik at tuhh.de Mon Feb 28 04:42:14 2005 From: s.malik at tuhh.de (Sireen Habib Malik) Date: Mon, 28 Feb 2005 13:42:14 +0100 Subject: [e2e] 10% packet loss stops TCP flow In-Reply-To: <4222D2C4.7060502@tuhh.de> References: <4222D2C4.7060502@tuhh.de> Message-ID: <422311A6.2050403@tuhh.de> for w= avg_window, is it w^2 * p =1.5? sorry to the sorry to be trivial. .. SM ireen Habib Malik wrote: > Hi, > > ......sorry to be trivial but please correct me if i think the > expression should be (w^2)*p=2.25. > > I would have worked out the answer the same way but through the above > expression. > > thanks for the clarification. > > -- > Sireen Malik > > Christian Huitema wrote: > >>> There is another very simple way to look at it. The most >>> "optimistic" model of TCP, that only looks at congestion >>> avoidance (AIMD) and ignores slow start, timeouts etc., predicts >>> the following relationship between the average window size (w) >>> and loss probability (p): >>> >>> (w^2)*p = 2 >>> >>> Note that this is _independent_ of link bandwidth and RTT. So >>> with a loss probability of 10%, the average window size is going >>> to be of the order 4. If you factor in other protocol details, >>> the number will be even smaller. With a window size of 4 or >>> below, fast recovery will not work and any loss will lead to a >>> timeout (for the average connection). Hence loss rates of the >>> order 10% simply will not work, irrespective of RTT/link bandwidth. >>> >> >> >> The formula (w^2)*p = 2 assumes a connection in "congestion avoidance" >> mode, which is not actually the case if the loss rate is high. If the >> loss rate is high, the connection will be very often in "slow start" >> mode, where the approximation is "w*p = 2". If the loss rate is still >> higher, we move to "w=1", and the adaptation comes from increasing the >> RTT. >> The classic rule of thumb is that a link exhibiting more than 1% packet >> loss is considered broken. >> >> By the way, Noel's challenge points to an interesting point. Long >> transfers are more likely to break than short transfers. If we want to >> actually transfer large files over poor quality networks, we cannot rely >> on TCP alone, but we have to split them in small chunks. Much like what >> BitTorrent does... >> >> -- Christian Huitema >> >> > > -- Sireen Malik, M.Sc. PhD. Candidate, Communication Networks Hamburg University of Technology, FSP 4-06 (room 3008) Denickestr. 17 21073 Hamburg, Deutschland Tel: +49 (40) 42-878-3387 Fax: +49 (40) 42-878-2941 E-Mail: s.malik at tuhh.de --Everything should be as simple as possible, but no simpler (Albert Einstein) From dpreed at reed.com Mon Feb 28 05:02:41 2005 From: dpreed at reed.com (David P. Reed) Date: Mon, 28 Feb 2005 08:02:41 -0500 Subject: [e2e] 10% packet loss stops TCP flow In-Reply-To: <4222FF04.9050204@tuhh.de> References: <20050225230810.1244E86AE5@mercury.lcs.mit.edu> <4222FF04.9050204@tuhh.de> Message-ID: <42231671.70104@reed.com> Interesting theoretical discussion, but remember that TCP was designed to run on networks with low packet loss rates, or networks that use link-level retransmission or FEC to get packet loss rates to less than 1%. This is what was meant by "best efforts". There've been a lot of people asking the question whether TCP can eliminate the value of link level protocols as optimizations. Of course it can't - a properly structured "attacker" can disrupt TCP by many techniques, and some of those can be viewed as plausible behaviors resulting from "unintentional" errors (intentional by the network designer or manager to leave them untreated or deploy them as middleboxes, though). This is a great discussion, but I hope no theoretician is confused enough to think that making TCP run on a 10% loss rate system is of practical interest. From craig at aland.bbn.com Mon Feb 28 05:30:10 2005 From: craig at aland.bbn.com (Craig Partridge) Date: Mon, 28 Feb 2005 08:30:10 -0500 Subject: [e2e] TCP losses In-Reply-To: Your message of "Mon, 28 Feb 2005 10:54:09 +0100." <4222EA41.2020201@tuhh.de> Message-ID: <20050228133010.849311FF@aland.bbn.com> In message <4222EA41.2020201 at tuhh.de>, Sireen Habib Malik writes: >Having said that it is also clear that considering the protocol without >considering the under-lying layers is not correct. Considering >"congestion" alone as cause of losses is not sufficient - the protocol >when zipping through layers underneath it, sees all kinds of failures. >Those failures which cause the TCP to have "serious" performance problems. > >Obviously one way to correct those failures is at their respective >layers. The other is to make TCP responsible for the 2*E2E loop for >both the data transport and fault-recovery functionality. Take a peek at Rainer Ludwig's doctoral dissertation on cross-layer interactions in wireless. It is a useful discussion and starting point. A useful observation is that, in certain situations, link layer retransmission is just the right thing to help TCP, but in high loss scenarios it can be wrong as TCP will retransmit and then you'll have the wireless layer ardently trying to retransmit two copies of the same segment... Craig From dpreed at reed.com Mon Feb 28 06:38:59 2005 From: dpreed at reed.com (David P. Reed) Date: Mon, 28 Feb 2005 09:38:59 -0500 Subject: [e2e] TCP losses In-Reply-To: <20050228133010.849311FF@aland.bbn.com> References: <20050228133010.849311FF@aland.bbn.com> Message-ID: <42232D03.2030509@reed.com> I'd suggest as another interesting input to "cross-layer" discussions the excellent lead article by PR Kumar on the topic in IEEE Wireless Communications Systems' latest issue. In many ways it parallels the "end-to-end argument" paper we wrote many years ago - beware of letting a too narrow view of the end-to-end problem drive you into creating an over-optimized, brittle, complex architecture that eliminates the value of modularizing design. But it stands on its own as a nice exploration of the issues in systems-scale wireless design. From limkt_2 at hotmail.com Mon Feb 28 09:09:36 2005 From: limkt_2 at hotmail.com (Lim Kong Teong) Date: Mon, 28 Feb 2005 17:09:36 +0000 Subject: [e2e] A question regarding loss event rate estimation Message-ID: Hi, I am sorry for asking this simple question, but I need to ascertain that I do it correctly. Let say, I am simulating a TFRC flow with deterministic periodic loss, and 4 packets are sent per RTT. I set the loss rate to 20%, that is for every 5 packets sent one packet will be dropped. So the drop pattern will be as follow: S S S S D S S S S D S S S S D S S S S D S S S S D S S S S D S = successful packet and D = dropped packet My question is how many packets in a loss interval: 4 or 5. My understanding loss interval size is the successful packets between two lost event, thus the size of a loss interval is 4. However, it is equal to 25% (1/4) loss rate - not 20%. Thank you. _________________________________________________________________ FREE Pocket Business English, ACT NOW! http://go.msnserver.com/HK/46165.asp From huitema at windows.microsoft.com Mon Feb 28 09:23:29 2005 From: huitema at windows.microsoft.com (Christian Huitema) Date: Mon, 28 Feb 2005 09:23:29 -0800 Subject: [e2e] 10% packet loss stops TCP flow Message-ID: > There've been a lot of people asking the question whether TCP can > eliminate the value of link level protocols as optimizations. This was hotly debated in the early 80's, part of the X.25 vs. TCP-IP debate, and a few more. The key variable is the number of bits in transit on the flow being controlled. This variable conditions the amount of memory required at each control point for retransmission and reordering buffers. The product of this variable by the loss rate is the "average number of errors in transit", and it conditions whether simple protocols can be used (e.g. go-back-N), or whether more sophisticated algorithms are required (e.g. selective ACK). The average number of error in transit also conditions the average retransmission delay (go-back-N) or the average reordering delay (selective ACK). The number of bits in transit is the product of the data rate by the transmission delays. If there was single flow in transit, hop-by-hop control would be more efficient than end-to-end, since the end-to-end delay is the sum of the link delay. However, there are usually multiple flows in transit, and the data-rate per end-to-end flow can be much less than the link data rate. As a rule of thumb, if the number of flows per hop exceeds the number of hops per flow, the bandwidth-delay product per average end-to-end flow will be less than the bandwidth-delay product per average link, and end-to-end control will result in better performance. -- Christian Huitema From fred at cisco.com Mon Feb 28 09:41:27 2005 From: fred at cisco.com (Fred Baker) Date: Mon, 28 Feb 2005 09:41:27 -0800 Subject: [e2e] TCP losses In-Reply-To: <4222EA41.2020201@tuhh.de> References: <4222EA41.2020201@tuhh.de> Message-ID: <6.2.1.2.2.20050228093632.05d7e3f0@mira-sjc5-b.cisco.com> At 10:54 AM 02/28/05 +0100, Sireen Habib Malik wrote: >How long does TCP take to decide that its connection has gone to the dogs >and that the Application must do something about it? RFC1122 (section >4.2.3.5) talks about "atleast" 100seconds. Is this applied practically? By some, yes, and often not. Practically, a routing outage lasting for tens of seconds often results in TCP sessions failing. I'm not sure I would peg it to 100 seconds nowadays, but some do seem a little brittle. >At this point, i am interested in knowing what breaks TCP outside its own >congestion related problems. What are those failures? How frequently they >occur? Any idea of duration? I would call those "loss-related", not "congestion-related". TCP only sees congestion in the form of loss or variation in RTT. >It would be nicer, if the errors relevant for future large bandwidth-delay >product IP over DWDM networks be given priority. I'm not sure I understand that statement. Are you asking responders to think about DWDM (because it is interesting to you), or wondering whether errors in a DWDM environment should be treated in some special way by TCP, or what? From s.malik at tuhh.de Mon Feb 28 10:38:02 2005 From: s.malik at tuhh.de (Sireen Habib Malik) Date: Mon, 28 Feb 2005 19:38:02 +0100 Subject: [e2e] TCP losses In-Reply-To: <6.2.1.2.2.20050228093632.05d7e3f0@mira-sjc5-b.cisco.com> References: <4222EA41.2020201@tuhh.de> <6.2.1.2.2.20050228093632.05d7e3f0@mira-sjc5-b.cisco.com> Message-ID: <4223650A.90600@tuhh.de> Fred Baker wrote: > At 10:54 AM 02/28/05 +0100, Sireen Habib Malik wrote: > >> How long does TCP take to decide that its connection has gone to the >> dogs and that the Application must do something about it? RFC1122 >> (section 4.2.3.5) talks about "atleast" 100seconds. Is this applied >> practically? > > > By some, yes, and often not. Practically, a routing outage lasting for > tens of seconds often results in TCP sessions failing. I'm not sure I > would peg it to 100 seconds nowadays, but some do seem a little brittle. > Thanks for the input. >> At this point, i am interested in knowing what breaks TCP outside its >> own congestion related problems. What are those failures? How >> frequently they occur? Any idea of duration? > > > I would call those "loss-related", not "congestion-related". TCP only > sees congestion in the form of loss or variation in RTT. > got it. >> It would be nicer, if the errors relevant for future large >> bandwidth-delay product IP over DWDM networks be given priority. > > > I'm not sure I understand that statement. Are you asking responders to > think about DWDM (because it is interesting to you), or wondering > whether errors in a DWDM environment should be treated in some special > way by TCP, or what? Yes, IP over DWDM is my area of work and am interested in knowing what issues to consider. To be more precise, i am simulating optical burst switching networks and congestion is one of the two thing that I have considered. All the rest of faults, i am simulating in a very crude manner by disrupting traffic for a X random time (presently deterministic). This way we can atleast study the impact of different protection and restoration on the application layer. I need more data to bring this crude-method closer to reality. regards, SM -- Sireen Malik, M.Sc. PhD. Candidate, Communication Networks Hamburg University of Technology, FSP 4-06 (room 3008) Denickestr. 17 21073 Hamburg, Deutschland Tel: +49 (40) 42-878-3387 Fax: +49 (40) 42-878-2941 E-Mail: s.malik at tuhh.de --Everything should be as simple as possible, but no simpler (Albert Einstein) From sommerfeld at sun.com Mon Feb 28 10:52:05 2005 From: sommerfeld at sun.com (Bill Sommerfeld) Date: Mon, 28 Feb 2005 13:52:05 -0500 Subject: [e2e] TCP losses In-Reply-To: <4222EA41.2020201@tuhh.de> References: <4222EA41.2020201@tuhh.de> Message-ID: <1109616724.23308.158.camel@thunk> On Mon, 2005-02-28 at 04:54, Sireen Habib Malik wrote: > How long does TCP take to decide that its connection has gone to the > dogs and that the Application must do something about it? RFC1122 > (section 4.2.3.5) talks about "atleast" 100seconds. Is this applied > practically? Most of the stacks I'm familiar with give an established connection at least that long; perhaps more like 5+ minutes. When there's a user waiting, they often decide to give up before TCP does.. > It would be nicer, if the errors relevant for future large > bandwidth-delay product IP over DWDM networks be given priority. I guess it depends on what part of the network you have to use on a regular basis. The thing I keep surprising myself with is how many differently-broken networks show improved performance when you shrink the tcp window. - Bill From braden at ISI.EDU Mon Feb 28 08:50:30 2005 From: braden at ISI.EDU (Bob Braden) Date: Mon, 28 Feb 2005 08:50:30 -0800 (PST) Subject: [e2e] ACM SIGCOMM 2005 Workshop on Mining Network Data (MineNet-05) Message-ID: <200502281650.IAA12144@gra.isi.edu> ------------------------------------------------------------------------ ---- Call for papers ****************************************************************** * * * SIGCOMM 2005 Workshop on Mining Network Data (MineNet-05) * * * * (to be held with SIGCOMM 2005, Aug 20-26, Philadelphia, USA) * * * * http://www.acm.org/sigs/sigcomm/sigcomm2005/cfp-minenet.html * ****************************************************************** Today's IP networks are extensively instrumented for collecting a wealth of information including traffic traces (e.g., packet or flow level traces), control (e.g., router forwarding tables, BGP and OSPF updates), and management (e.g., alarms, SNMP traps) data. The real challenge is to process and analyze this vast amount of primarily unstructured information and extract structures, relationships, and "higher level knowledge" embedded in it and use it to aid network management and operations. The goal of this one day workshop is to explore new directions in network data collection, storage, and analysis techniques, and their application to network monitoring, management, and remediation. The workshop will provide a venue for researchers and practitioners from different backgrounds, including networking, data mining, machine learning, and statistics, to get together and collaboratively approach this problem from their respective vantage points. We are soliciting original/position papers on topics (including, but not limited to) listed below. - Collection, storage & access infrastructure: platform instrumentation (e.g. multi-modal, multi-resolution sensors), collection techniques (e.g. event sampling, filtering, aggregation, etc.), storage and access (e.g. retention policy, indexing techniques etc.). - Network data analytics techniques & tools: network stream mining, network graph mining, micro-clustering, temporal and statistical correlation, causality tracking, machine learning. - Applications to network operations & management: network problem determination, network reliability and performance, root-cause analysis, security, emerging phenomenon detection (e.g. DDoS, virus/worm, spam etc.), traffic classification. Of particular interest are (i) new solution techniques as well as applications of existing techniques from data mining, machine learning and statistics to IP network problems, (ii) experiences with the use of such techniques for IP networks, and (iii) open networking problems and challenges that would benefit from the use of such techniques. Selected papers will be forward-looking, with impact and implications for both operational networks and ongoing or future research. Submission Instructions ----------------------- Papers should be at most 6 pages long, in standard ACM format (single-spaced, double column, at least 10pt font). Accepted papers will appear in the workshop proceedings. Authors of accepted papers are expected to present their work at the workshop. Detailed submission guidelines will be available soon at http://www.acm.org/sigs/sigcomm/sigcomm2005/cfp-minenet.html. Important Dates --------------- Submission Deadline: Wed, April 6, 2005, 9.00 PM EST Notification Deadline: May 10, 2005 Camera Ready Deadline: May 30, 2005 Workshop Date: Aug 26, 2005 Workshop Organizing Chairs -------------------------- Subhabrata Sen, AT&T Research Chuanyi Ji, Georgia Tech Debanjan Saha, IBM Research Joe McCloskey, Dept. of Defense Technical Program Committee ---------------------------- Constantinos Dovrolis, Georgia Tech. Chuanyi Ji, Georgia Tech. Nick Koudas, Univ. of Toronto Vipin Kumar, Univ. of Minnesota Joe McCloskey, Dept. of Defense Robert Novak, Univ. of Wisconsin-Madison Rajeev Rastogi, Lucent Bell Labs John Reumann, IBM Research Jennifer Rexford, Princeton Univ. Mathew Roughan, Univ. of Adelaide Debanjan Saha, IBM Research Subhabrata Sen, AT&T Research William Szewczyk, Dept. of Defense Walter Willinger, AT&T Research Zhi-Li Zhang, Univ. of Minnesota ----- End Included Message -----