From Cheng.Huang at microsoft.com Fri Feb 4 10:11:08 2011 From: Cheng.Huang at microsoft.com (Cheng Huang) Date: Fri, 4 Feb 2011 18:11:08 +0000 Subject: [e2e] [CFP] HotMD 2011 -- Hot Topics in Multimedia Delivery Message-ID: HotMD 2011 -- Hot Topics in Multimedia Delivery Barcelona, Spain, July 2011 Deadline: Feb. 20, 2011 http://sites.google.com/site/hotmd2011/ Workshop co-chairs * Cheng Huang, Microsoft Research, USA * Yong Liu, Polytechnic Institute of NYU, USA * Francesc Pinyol Margalef, La Salle - Universitat Ramon Llull, Spain * Gabriel Fern?ndez Ubiergo, La Salle - Universitat Ramon Llull, Spain * Alejandro L?pez Gonz?lez, La Salle - Universitat Ramon Llull, Spain Program committee * Federico Alvarez, Polytechnic University of Madrid * Yunnan Wu, Facebook * Wei Tsang Ooi, National University of Singapore * Antonio Navarro, Instituto de Telecomunica??es * Songqing Chen, George Mason University * Yang Guo, Bell Labs * Zhi-Li Zhang, University of Minnesota * Ali Begen, Cisco * Jesus Alcober, Polytechnic University of Catalonia * Baochun Li, University of Toronto * Thomas Schierl, Fraunhofer HHI - Heinrich Hertz Institute * Stewart T. Worrall, University of Surrey * David G?mez-Barquero, Polytechnic University of Valencia * Zhu Liu, AT&T Research Labs * John Lui, Chinese University of Hong Kong * Bo Li, Hong Kong University of Science and Technology * Jiangchuan Liu, Simon Fraser University * Anees Shaikh, IBM Research * Richard Yang, Yale University * Sanjeev Mehrotra, Microsoft Research * Sanjay Rao, Purdue University * Georg Thallinger, Joanneum Research * Andy Berkheimer, Google -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20110204/7b5ec594/attachment.html From ratul at microsoft.com Mon Feb 7 09:55:39 2011 From: ratul at microsoft.com (Ratul Mahajan) Date: Mon, 7 Feb 2011 17:55:39 +0000 Subject: [e2e] CFP: MobiArch 2011 Message-ID: <1E62F7447986F548BDC0B986CC69B8514C636498@TK5EX14MBXC120.redmond.corp.microsoft.com> Call for Papers: The 6th ACM International Workshop on Mobility in the Evolving Internet Architecture (MobiArch 2011) http://mobiarch11.cs.ucl.ac.uk June 28, 2011 Co-located with ACM MobiSys 2011 in Washington, D.C. Technical Area With recent advances in technologies for wireless access (e.g., WiFi, 3G, and 4G) and mobile devices (e.g., smartphones, netbooks, and tablets), mobility has become a fundamental characteristic of today's Internet. Yet, basic architectural issues related to mobility such as efficient mobility management, the locator-identifier split, multi-homing, security, and various operational, deployment concerns are still in their early stages of exploration. Moreover, the Internet architecture itself, its end-to-end principles, and business models require rethinking due to the massive penetration of mobility into the Internet. For instance, an appropriate system that allows communicating with a mobile host requires addressing several fundamental issues with the Internet architecture, such as ability to locate the mobile host/service, preserving ongoing communications upon changes of locations, as well as seamless and secure handover management. As another example, the emerging wireless technologies such as those in the 60 GHz and whitespace bands may pose additional challenges since they may introduce design principles different from the packet-switched Internet. MobiArch 2011 welcomes submissions, from both researchers and practitioners, in exploration of recent advances in architectures, protocols, and experiences with emerging technologies on various mobility issues over the Internet, with an emphasis on wireless infrastructures and mobility patterns for mobility support, new mobility protocols, service discovery, routing and location management, mobile network performance evaluation and modelling, multi-homing, security, architectural impacts and deployment considerations. We encourage submission of early, in-progress work as well as position papers. The workshop will address all architectural issues and system support for mobility in the Internet, including but not limited to: * Impacts of new wireless technologies/services, networking technologies, and mobility patterns on Internet architecture. * Architectures and protocols for mobility support in the Internet, ranging from approaches in the link, network, and transport layers, to the application layer and cross-layer design. * Architectures and protocols for service partitioning, code offloading, and data center management to support mobile devices in the compute cloud. * Location management, robustness, routing, locator/identifier split, multi-homing and load sharing issues. * Security, dependability, and privacy issues in mobility networks, and their impacts on Internet architecture. * Architectures and mechanisms for wireless/mobile connectivity in remote areas and developing countries. * Performance issues with mobility in the Internet. * QoS and middlebox issues in mobility networks and impacts on Internet architecture. * Economic and deployment issues of mobility solutions (infrastructure and devices). * Impact of social aspects on mobility architectures, mobile application and protocol design. Paper format and submission instructions Submitted papers must be no more than 6 pages long, with no characters in smaller than 10 point fonts. Paper submission will be handled via HotCRP. Papers will be reviewed single-blind. Important Dates Submission Deadline: March 1, 2011, 11:59PM Eastern Standard Time (EST) Acceptance Notification: March 31, 2011 Camera Ready Due: April 24, 2011 Program committee Kyle Jamieson, University College London, UK (co-chair) Ratul Mahajan, Microsoft Research Redmond, USA (co-chair) Ashok Agrawala, University of Maryland, USA Rajesh Balan, Singapore Management University, Singapore Nilanjan Banerjee, University of Arkansas, USA Olivier Bonaventure, Universit? Catholique de Louvain, Belgium Ranveer Chandra, MSR Redmond, USA Jakob Eriksson, University of Illinois at Chicago, USA Jaeyeon Jung, Intel Labs Seattle, USA Li Li, Bell Labs, USA Costin Raiciu, Politehnica U. of Bucharest, Romania Dipankar Raychaudhuri, Rutgers University, USA Injong Rhee, NC State University, USA Mahadev Satyanarayan, CMU, USA Jonathan Smith, University of Pennsylvania, USA Alex Snoeren, UCSD, USA Oliver Spatscheck, AT&T Labs, USA Peter Steenkiste, CMU, USA Kun Tan, MSR Asia, USA Mike Walfish, UT Austin, USA Lixia Zhang, UC Los Angeles, USA Steering Committee Jon Crowcroft, University of Cambridge, UK (chair) Xiaoming Fu, University of Goettingen, Germany (ex-officio) Katherine Guo, Bell Laboratories, USA Henning Schulzrinne, Columbia University, USA Lars Eggert, Nokia Research Center, Finland J?rg Ott, Helsinki University of Technology, Finland Marco Gruteser, Rutgers University, USA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20110207/63a7ed72/attachment.html From Renata.Teixeira at lip6.fr Tue Feb 15 19:11:55 2011 From: Renata.Teixeira at lip6.fr (Renata.Teixeira@lip6.fr) Date: Wed, 16 Feb 2011 04:11:55 +0100 Subject: [e2e] CFP: ACM Internet Measurement Conference 2011 Message-ID: <20110216041155.5764720d04l74uwr@webmail.lip6.fr> --------------------------------------------------------------------- Call for Papers Internet Measurement Conference 2011 Sponsored by ACM SIGCOMM and ACM SIGMETRICS and in cooperation with USENIX November 2-4, 2011 Berlin, Germany http://conferences.sigcomm.org/imc/2011 --------------------------------------------------------------------- The eleventh Internet Measurement Conference is a three day event focusing on all aspects of measurement-based research pertaining to today's Internet. Building on the success of past IMCs, we invite submissions of papers that contribute to our understanding of all aspects related to the Internet through the collection, analysis, or visualization of all types of network-related measurements. Examples of relevant topics are: * Internet traffic and topology measurement * Internet-oriented wireless, and mobility measurement * Internet performance measurements * Inter-domain and intra-domain routing * Measurement-based network management such as traffic engineering * Network applications such as multimedia streaming, gaming and on-line social networks * Measurement of sensing applications, including home networking and green networking * Measurements of content distribution, peer-to-peer, overlay, and social networks * Internet data-specific issues, including anonymization, querying, and storage * Measurement-based inference of network properties and structures * Design of and experiences with new monitoring systems * Algorithms for network measurements, including sampling and statistical inference methods * Network anomaly detection and troubleshooting * Software tools for analyzing and visualizing high-volume Internet- related datasets * Measurement related to network security and privacy * Measurement-based assessment of simulation/testbeds * Measurement-based modeling and workload generation * Reappraisal of previous measurement findings * Measurement related to data centers and the cloud * Measurement of technological and socio-economic aspects of networking * Relevant end-host and end-user measurements Papers that do not in some fashion relate to measuring aspects of the Internet or Internet-like systems are out of scope. Moreover, papers describing original ideas and novel validation methods will be given preference over submissions that report on incremental improvements of the existing state-of-the-art. Authors can contact the Program Co-Chairs for clarification if they are unsure whether their paper is in scope. Ethical standards for measurement must be considered by all IMC authors. In particular, authors must be aware of and conform to acceptable use policies for individual domains that are probed or monitored, data privacy and anonymity for all personally identifiable information, and etiquette for using shared measurement data (see Allman and Paxson, IMC '07). If applicable, authors are also urged to notify parties of security flaws in their products or services in advance of publication. Adherence to ethical standards for measurement will be a criteria for all submissions and violations---including ambiguous situations not well described---will be grounds for rejection. Submission Guidelines --------------------- There are two forms of submissions: 1. Full papers (up to 14 two-column pages) describing original research, with succinctness appropriate to the topics and themes they discuss. 2. Short papers (up to 6 two-column pages for text and figures + 1 page for references) conveying work that is less mature but shows promise, articulating a high-level vision, describing challenging future directions, critiquing current measurement wisdom or offering results that do not merit a full submission. Short papers will be subject to a 7-page limit in the Proceedings (with the last page for references only). Note: Previous IMCs have utilized a "reject to short" notion whereby full paper submissions have been accepted as short papers. IMC 2011 will not be using this procedure. Any submission longer than allowed by the above short guidelines will be considered a full paper. Note: Authors are encouraged to think carefully about whether to submit a full or short paper. In the past there have been many instances whereby a full submission has been found to be lacking enough technical contribution for a full paper, but for which the PC finds an interesting portion of the paper that would have been a very nice contribution as a short paper. Submissions must be in electronic form, as PDF documents. The submission must conform to the page limits stated above, and with text written in at least a 10-point font (Fonts used in Figures etc should be no smaller than 9 pt) satisfying the requirements specified below. * use double column format * the size of each column should be at most 9.25" by 3.33" * the space between columns should be at least 0.33" * use 10pt font * use up to 55 lines of text per column The http://conferences.sigcomm.org/imc/2009/sig-alternate-10pt.cls style file should satisfy these requirements. Please use -t letter when converting the .dvi file to pdf. All manuscripts must be in English and do not need to be anonymized. Submissions that do not comply with these requirements will not be read. All full papers and short papers accepted for presentation will be published in the Conference Proceedings produced by ACM. A few accepted papers may be forwarded for fast-track submission to the IEEE/ACM Transactions on Networking. To encourage broader data sharing in the community, the conference will present a best paper award for the top paper that makes its data sets publically available by the time of camera ready submission. For example, wireless-network data sets may be published through CRAWDAD. Authors that would like their paper to be considered for this award should add a footnote on the first page of their submission (and indicate this by selecting the appropriate button on the paper registration/submission page). A limited number of travel grants may be available to students who are unable to secure funding from their advisors. Important Dates --------------- * May 6, 2011, mid-night EDT: Registration of title and 250-word abstract * May 13, 2011, mid-night EDT: Hard submission deadline * July 22, 2011: Notification * November 2-4, 2011: Conference held in Berlin, Germany Program Co-Chairs ----------------- Patrick Thiran, EPFL Walter Willinger, AT&T Labs-Research Program Committee ----------------- Katerina Argyraki (EPFL, Switzerland) Martin Arlitt (HP Labs and University of Calgary, Canada) Suman Banerjee (University of Wisconsin, USA) Bobby Bhattacharjee (University of Maryland, USA) Fabian Bustamante (Northwestern University, USA) Augustin Chaintreau (Columbia University, USA) Xenofontas Dimitropoulos (ETH Zurich, Switzerland) Lixin Gao (University of Massachusetts, USA) Alexandre Gerber (AT&T Labs-Research, USA) Srikanth Kandula (Microsoft Research, USA) Thomas Karagiannis (Mirosoft Research, UK) Sachin Katti (Stanford University, USA) Ramana Kompella (Purdue University, USA) Christian Kreibich (ICSI, USA) Nikolaos Laoutaris (Telefonica Research, Spain) Olaf Maennel (Loughborough University, UK) Ratul Mahajan (Microsoft Research, USA) Z. Morley Mao (University of Michigan, USA) Hung X. Nguyen (University of Adelaide, Australia) Antonio Nucci (Narus, USA) Konstantina (Dina) Papagiannaki (Intel Labs, USA) Lili Qiu (University of Texas, USA) Reza Rejaie (University of Oregon, USA) Pablo Rodriguez (Telefonica Research, Spain) Matthew Roughan (University of Adelaide, Australia) Vyas Sekar (Intel Labs, USA) Rob Sherwood (Deutsche Telekom Labs, USA) Georgios Smaragdakis (TU Berlin and Deutsche Telekom Labs, Germany) Joel Sommers (Colgate University, USA) Neil Spring (University of Maryland, USA) Renata Teixeira (CNRS and UPMC Sorbonne Universites, France) Steve Uhlig (TU Berlin and Deutsche Telekom Labs, Germany) David Wetherall (University of Washington, USA) Lixia Zhang (UCLA, USA) Sponsors -------- Gold: Akamai Silver: HP and Technicolor From queszama at yahoo.in Tue Feb 22 01:24:31 2011 From: queszama at yahoo.in (Zama Ques) Date: Tue, 22 Feb 2011 14:54:31 +0530 (IST) Subject: [e2e] query on behaviour of tcp_keepalive and tcp retransmit on Linux based systems Message-ID: <449322.18998.qm@web137509.mail.in.yahoo.com> We need some clarifications on TCP_keepalive .? We are facing some issues on our Prod servers related to TCP functionality . The issue is like this. We have some machines at one end sending data in real time to another group of machines on the other hand .? Now due to some hardware issues on the other hand , some of the machines becomes unresponsive/crashes. The client system which pumps data never came to know that the server went unresponsive . The connection remains in ESTABLISHED state and the client always tries to send data thinking that the connection is alive because of which we are seeing backlog on client sides. Our understanding is like this on how TCP will handle the connection. Q 1) Since? the server went down , the client will try to the retransmit the data until it times out. What is the behavior of TCP after the timeout? Need clarification on the following things. a) Will the kernel will close the established connection after the timeout . Looks like no in our case as we still see the connection still in ESTABLISHED state after around more than 2 hours. b) Are there any kernel parameters which decides the when the client is timeout after retransmission fails. What is the behavior of TCP after the client retransmission timeouts. Q 2 ) There is something called tcp_keepalive which if implemented in the kernel , by default it's there and comes to be around 2 hrs 2 minsutes , i think? ,? the client will send some TCP probes after the keepalive time ineterval and if it cannot reach the server , then the established connection in the client side will be closed by the kernel . This is my understanding. But I can see that the connection still remains in established after the tcp_keepalive time . We waited for around 2 hrs 30 minutes but the connection remains in established state only. Tried reducing the keepalive time to be around 10 minutes , but the connection remains in ESTABLISHED state in client side . Where I went wrong .Please clarify my doubts raised above . What should we do to resolve the problem we are seeing above . Any help will be highly appreciated as we are going through a hard time to resolve the issue . Thanks in Advance -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20110222/39f64409/attachment.html From ycai at ecs.umass.edu Tue Feb 22 06:55:13 2011 From: ycai at ecs.umass.edu (Yan Cai) Date: Tue, 22 Feb 2011 09:55:13 -0500 Subject: [e2e] query on behaviour of tcp_keepalive and tcp retransmit on Linux based systems In-Reply-To: <449322.18998.qm@web137509.mail.in.yahoo.com> References: <449322.18998.qm@web137509.mail.in.yahoo.com> Message-ID: <4D63CE50.8050606@ecs.umass.edu> Hi According to your description, the expected behavior should be as follows. At the beginning senders at one side can send data to the receivers at the other side, and the receivers can receive data without any problem. When some of the receivers become off-line, the affected senders should no long receive positive acknowledgments, therefore, lowering their congestion windows (i.e., sending rate). Since in your case the receiver is off forever, some senders should further experience timeout events. After a few timeouts, the sender should CLOSE this connection itself. As far as I know, the whole procedure above should be automatically invoked in the sender side. This is how TCP (sender) handles exceptions. My suggestion is that you run a simple experiment on your side to see if TCP in your machine can work that way. The test can be done using i-perf to send a long long live TCP flow, and then take off the receiver in the middle of the transmission. The connection is expected to be closed very soon after the receiver is off. Hope it helpful. Yan On 2/22/2011 4:24 AM, Zama Ques wrote: > We need some clarifications on TCP_keepalive . We are facing some > issues on our Prod servers related to TCP functionality . > > The issue is like this. > > We have some machines at one end sending data in real time to another > group of machines on the other hand . Now due to some hardware issues > on the other hand , some of the machines becomes unresponsive/crashes. > The client system which pumps data never came to know that the server > went unresponsive . The connection remains in > ESTABLISHED state and the client always tries to send data thinking > that the connection is alive because of which we are seeing backlog on > client sides. > > Our understanding is like this on how TCP will handle the connection. > > > Q 1) Since the server went down , the client will try to the > retransmit the data until it times out. What is the behavior of TCP > after the timeout? Need clarification on > the following things. > a) Will the kernel will close the established connection after the > timeout . Looks like no in our case as we still see the connection > still in ESTABLISHED state after around more > than 2 hours. > b) Are there any kernel parameters which decides the when the client > is timeout after retransmission fails. What is the behavior of TCP > after the client retransmission timeouts. > > > Q 2 ) There is something called tcp_keepalive which if implemented in > the kernel , by default it's there and comes to be around 2 hrs 2 > minsutes , i think , the client will send some TCP probes after the > keepalive time ineterval and if it cannot reach the server , then the > established connection in the client side will be closed by the kernel > . This is my understanding. But I can see that the connection still > remains in established after the tcp_keepalive time . We waited for > around 2 hrs 30 minutes but the connection remains in established > state only. Tried reducing the keepalive time to be around 10 minutes > , but the connection remains in ESTABLISHED state in client side . > > > Where I went wrong .Please clarify my doubts raised above . What > should we do to resolve the problem we are seeing above . Any help > will be highly appreciated as we are going through a hard time to > resolve the issue . > > Thanks in Advance > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20110222/50be8540/attachment.html From nweng at siu.edu Mon Feb 21 09:03:06 2011 From: nweng at siu.edu (Ning Weng) Date: Mon, 21 Feb 2011 11:03:06 -0600 Subject: [e2e] ANCS CFP Message-ID: <003a01cbd1e9$366a4370$a33eca50$@edu> CALL FOR PAPERS The 7th ACM/IEEE Symposium on Architectures for Networking and Communications Systems ANCS 2011 http://www.ancsconf.org October 3-4, 2011 Polytechnic Institute of NYU Brooklyn, New York, USA Sponsored by (pending): ACM Special Interest Group on Computer Architecture (SIGARCH) ACM Special Interest Group on Communications (SIGCOMM) IEEE Computer Society Tech. Committee on Computer Architecture (TCCA) IMPORTANT DATES: Paper registration and abstract: May 9, 2011 Submission deadline: May 16, 2011 Author notification: July 29, 2011 CONFERENCE OVERVIEW: ANCS is a systems-oriented research conference, presenting original work that explores the relationship between the architecture of modern computer networks and the architecture of the individual hardware and software elements from which these networks are built. This year's conference will emphasize in its paper selection research on computer and network systems that provide the foundations of emerging network technologies and the future Internet. Topics of interest include, but are not limited to: * System design for future network architectures * Network architectures enabled by converged platforms * Virtualized infrastructure architectures, rationale, and devices * Converged router, server, and storage platforms * Content-centric architectures, platforms, and mechanisms * Scalable programming and application frameworks * High-performance / high-function packet processing platforms * Power- and size-optimized computer and communications platforms * High-speed networking mechanisms and algorithms * Network security architectures and security anchor/enhancement devices * Single-chip platform integration * Network measurement techniques, architectures, and devices * Techniques and systems for large-scale data analysis * Host-network interface issues * Architectures for data centers and cloud computing * Router and switch architectures * Network on chip * Software radios * Wireless-networking hardware and related software SUBMISSION PROCEDURES: Papers must be registered, with a title and abstract, no later than May 9, 2011 at 11:59PM EST (US). The deadline for submissions of full papers is May 16, 2011 at 11:59PM EST (US). There will be no extensions of this deadline. Please see the conference web site for detailed submission guidelines Notification of Acceptance: July 29, 2011 The Program Committee may choose to accept certain papers conditionally, based on a shepherding process. SUBMISSION REQUIREMENTS: ANCS 2011 will use single-blind reviewing, so submitted papers should include the authors' names. Submit papers using a 10pt font on 12pt leading formatted for printing on Letter-sized (8.5" by 11") paper. Paper text blocks must follow ACM guidelines: double-column, with each column 9.25" by 3.33", 0.33" space between columns. Each column must contain no more than 55 lines of text. Latex and Word templates, and additional submission requirements, are available via . ANCS submissions must be no longer than 12 pages. The PC may choose to reject any paper that violates these format requirements, and no deadline extensions will be allowed for re-formatting. We encourage submissions containing original ideas that are relevant to the scope of ANCS. Like other conferences, ANCS requires that papers not be submitted simultaneously to any other conferences or publications; that submissions not be previously published in peer-reviewed conferences or journals; Papers describing work that was previously published in a peer- reviewed workshop are allowed, if the authors clearly describe what significant new content has been included. All submissions will be treated as confidential prior to publication on the ANCS 2011 Web site; rejected submissions will be permanently treated as confidential. BEST PAPER AWARD AND FAST-TRACK TO TRANSACTIONS ON NETWORKING ANCS 2011 will feature a best paper award, which will be announced at the conference. Some of the accepted papers will be fast-tracked as submissions to the ACM/IEEE Transactions on Networking. POSTER SESSION: ANCS 2011 will include a poster session. Submission deadlines and guidelines will be announced at a later date on the conference web site . GENERAL CHAIR: H. Jonathan Chao, Polytechnic Institute of NYU PROGRAM CHAIRS: Jose Duato, Polytechnic University of Valencia Tilman Wolf, University of Massachusetts, Amherst FINANCE CHAIR: N. Sertac Artan, Polytechnic Institute of NYU PUBLICITY CHAIR: Maria Engracia Gomez, Polytechnic University of Valencia Bin Liu, Tsinghua University Ning Weng, Southern Illinois University at Carbondale CONTACT INFORMATION: For questions, please contact the ANCS 2011 program chairs. --------------------------------------------- Ning Weng, Assistant Professor Dept. of Electrical & Computer Engineering Southern Illinois University Carbondale Office: Engineering E-119 Phone: (618) 453-7645 Fax: (618) 453-7972 Email: nweng at siu.edu WWW: http://www.engr.siu.edu/~weng --------------------------------------------- From queszama at yahoo.in Tue Feb 22 22:52:04 2011 From: queszama at yahoo.in (Zama Ques) Date: Wed, 23 Feb 2011 12:22:04 +0530 (IST) Subject: [e2e] end2end-interest Digest, Vol 83, Issue 4 In-Reply-To: Message-ID: <815402.87049.qm@web137507.mail.in.yahoo.com> Hi Yan, Thanks for your suggestion .? I am familiar with iperf but the issue with us that it is a prod network and it is advisable for me not to pump data on the network . Will try to the experiment between two desktops connected by a cross over cable. What I was trying earlier was that I started FTP server on one end? and connected to the server from the client side. ?$ ftp 10.66.X.X Connected to 10.66.X.X 220 (vsFTPd 2.2.2) Name (10.66.74.141:zama): anonymous 331 Please specify the password. Password: 230 Login successful. Remote system type is UNIX. Using binary mode to transfer files. After that I disconnected the network cable from the server and was monitoring the status of the connection on the client side . The status of the connection was like this before and after disconnecting the network cable. --- $? for i in {1..1000} ; do netstat -at | egrep "ftp" ; date? ; sleep 60? ; done tcp??????? 0????? 0 edgebeauty.c:50179 shopfrobsoon.c:ftp ESTABLISHED Wed Feb 23 11:47:53 IST 2011 tcp??????? 0????? 0 edgebeauty.c:50179 shopfrobsoon.c:ftp ESTABLISHED Wed Feb 23 11:48:53 IST 2011 tcp??????? 0????? 0 edgebeauty.c:50179 shopfrobsoon.c:ftp ESTABLISHED Wed Feb 23 11:49:53 IST 2011 ... ... Wed Feb 23 12:14:03 IST 2011 tcp??????? 0????? 0 edgebeauty.c:50179 shopfrobsoon.c:ftp ESTABLISHED Wed Feb 23 12:15:03 IST 2011 === If we see that the time is more than 25 minutes when the server went down and the client has still maintained the connection in established state. My understanding is that the client should close the connection after TCP restarsmit timeout happens or my understanding is wrong. Please clarify . --Zaman Message: 2 Date: Tue, 22 Feb 2011 09:55:13 -0500 From: Yan Cai Subject: Re: [e2e] query on behaviour of tcp_keepalive and tcp ??? retransmit on??? Linux based systems To: end2end-interest at postel.org Message-ID: <4D63CE50.8050606 at ecs.umass.edu> Content-Type: text/plain; charset="iso-8859-1" Hi According to your description, the expected behavior should be as follows. At the beginning senders at one side can send data to the receivers at the other side, and the receivers can receive data without any problem. When some of the receivers become off-line, the affected senders should no long receive positive acknowledgments, therefore, lowering their congestion windows (i.e., sending rate). Since in your case the receiver is off forever, some senders should further experience timeout events. After a few timeouts, the sender should CLOSE this connection itself. As far as I know, the whole procedure above should be automatically invoked in the sender side. This is how TCP (sender) handles exceptions. My suggestion is that you run a simple experiment on your side to see if TCP in your machine can work that way. The test can be done using i-perf to send a long long live TCP flow, and then take off the receiver in the middle of the transmission. The connection is expected to be closed very soon after the receiver is off. Hope it helpful. Yan On 2/22/2011 4:24 AM, Zama Ques wrote: > We need some clarifications on TCP_keepalive .? We are facing some > issues on our Prod servers related to TCP functionality . > > The issue is like this. > > We have some machines at one end sending data in real time to another > group of machines on the other hand .? Now due to some hardware issues > on the other hand , some of the machines becomes unresponsive/crashes. > The client system which pumps data never came to know that the server > went unresponsive . The connection remains in > ESTABLISHED state and the client always tries to send data thinking > that the connection is alive because of which we are seeing backlog on > client sides. > > Our understanding is like this on how TCP will handle the connection. > > > Q 1) Since? the server went down , the client will try to the > retransmit the data until it times out. What is the behavior of TCP > after the timeout? Need clarification on > the following things. > a) Will the kernel will close the established connection after the > timeout . Looks like no in our case as we still see the connection > still in ESTABLISHED state after around more > than 2 hours. > b) Are there any kernel parameters which decides the when the client > is timeout after retransmission fails. What is the behavior of TCP > after the client retransmission timeouts. > > > Q 2 ) There is something called tcp_keepalive which if implemented in > the kernel , by default it's there and comes to be around 2 hrs 2 > minsutes , i think? ,? the client will send some TCP probes after the > keepalive time ineterval and if it cannot reach the server , then the > established connection in the client side will be closed by the kernel > . This is my understanding. But I can see that the connection still > remains in established after the tcp_keepalive time . We waited for > around 2 hrs 30 minutes but the connection remains in established > state only. Tried reducing the keepalive time to be around 10 minutes > , but the connection remains in ESTABLISHED state in client side . > > > Where I went wrong .Please clarify my doubts raised above . What > should we do to resolve the problem we are seeing above . Any help > will be highly appreciated as we are going through a hard time to > resolve the issue . > > Thanks in Advance > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20110222/50be8540/attachment-0001.html ------------------------------ _______________________________________________ end2end-interest mailing list end2end-interest at postel.org http://mailman.postel.org/mailman/listinfo/end2end-interest End of end2end-interest Digest, Vol 83, Issue 4 *********************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20110223/e1f94dc0/attachment.html From queszama at yahoo.in Thu Feb 24 03:06:18 2011 From: queszama at yahoo.in (Zama Ques) Date: Thu, 24 Feb 2011 16:36:18 +0530 (IST) Subject: [e2e] end2end-interest Digest, Vol 83, Issue 4 In-Reply-To: <4D6513F8.1060903@ecs.umass.edu> Message-ID: <564728.78836.qm@web137502.mail.in.yahoo.com> Hi Yan, I tried testing with iperf today. Started server on one side and connected to the client from another host and after sometime disconnected the cable on the server host. Also? , reduced tcp_keepalive to 1200 sec , so timeout value should be like 32 minutes with the other two tcp_keepalive related kernel parameter (probes and interval) . The following are my findings . I can see that client terminates the ESTABLISHED connection after around 16 minutes since the server is not reachable , that is before the TCP keepalive timeout. Looks to me like this minutes is somehow related to TCP retransmission timeout which probably is determined by the following 3 parameters which comes to be around 18 minutes. . $ cat /proc/sys/net/ipv4/tcp_retries1 3 $ cat /proc/sys/net/ipv4/tcp_retries2 15 $ cat /proc/sys/net/ipv4/tcp_fin_timeout 60 Is my assumption correct here ? The following is the netstats connection flow during my experiment $? for i in {1..1000} ; do netstat -atn | egrep "5001" ; date? ; sleep 60? ; done tcp??????? 0 447432 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED tcp??????? 0????? 0 10.66.X.Y:43531??????????? 10.66.A.B:5001?????????? TIME_WAIT?? Thu Feb 24 14:47:16 IST 2011 tcp??????? 0 3311576 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 14:48:16 IST 2011 (Network Cable removed during this time from the server) tcp??????? 0 3317368 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 14:49:16 IST 2011 tcp??????? 0 3021976 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 14:50:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 14:51:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 14:52:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 14:53:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 14:54:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 14:55:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 14:56:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 14:57:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 14:58:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 14:59:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 15:00:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 15:01:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 15:02:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 15:03:16 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED Thu Feb 24 15:04:17 IST 2011 tcp??????? 0 2511048 10.66.X.Y:43533??????????? 10.66.A.B:5001?????????? ESTABLISHED .. (comes out be arnd 15 minutes from the server went unreachable when the connection status changed in client)? Thu Feb 24 15:05:17 IST 2011 Thu Feb 24 15:06:17 IST 2011 Thu Feb 24 15:07:17 IST 2011 The following packet flow can be seen on client as sniffed by tcpdump during while I removed the network cable . ===== 14:50:20.158794 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1801285561:1801350721, ack 0, win 92, options [nop,nop,TS val 172872553 ecr 177048109], length 65160 14:50:20.164550 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1801350721:1801415881, ack 0, win 92, options [nop,nop,TS val 172872558 ecr 177048115], length 65160 14:50:20.394916 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 172872992 ecr 177048116], length 1448 14:50:21.258921 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 172873856 ecr 177048116], length 1448 14:50:22.986922 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 172875584 ecr 177048116], length 1448 14:50:26.442922 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 172879040 ecr 177048116], length 1448 14:50:33.354923 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 172885952 ecr 177048116], length 1448 14:50:47.178932 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 172899776 ecr 177048116], length 1448 14:51:14.826929 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 172927424 ecr 177048116], length 1448 14:52:10.122922 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 172982720 ecr 177048116], length 1448 14:54:00.714934 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 173093312 ecr 177048116], length 1448 14:56:00.714921 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 173213312 ecr 177048116], length 1448 14:58:00.714920 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 173333312 ecr 177048116], length 1448 15:00:00.714921 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 173453312 ecr 177048116], length 1448 15:02:00.714921 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 173573312 ecr 177048116], length 1448 15:04:00.714936 IP edgebeauty-dr.43533 > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, ack 0, win 92, options [nop,nop,TS val 173693312 ecr 177048116], length 1448 ?Does my TCP stack look fine based on the experiments above . Thanks Zaman --- On Wed, 23/2/11, Yan Cai wrote: From: Yan Cai Subject: Re: end2end-interest Digest, Vol 83, Issue 4 To: "Zama Ques" Date: Wednesday, 23 February, 2011, 2:04 PM Hi Zaman, I guess there might be some unknown ftp configuration at CLIENT side that causes this issue. You can isolate the problem first. I-perf can be used to test functionality of tcp stack on your machine. If it works as expected, then there is nothing wrong with tcp stack. Next check the settings of the ftp client (not the ftp server) to see if there is any specific configuration that causes this problem. If it is hard to do that, my suggestion is to install a third party ftp client application and test with that. If none of them works, you might have to trace the traffic over the cable attached to the client machine and determine what is going on. Best wishes, Yan On 2/23/2011 1:52 AM, Zama Ques wrote: Hi Yan, Thanks for your suggestion .? I am familiar with iperf but the issue with us that it is a prod network and it is advisable for me not to pump data on the network . Will try to the experiment between two desktops connected by a cross over cable. What I was trying earlier was that I started FTP server on one end? and connected to the server from the client side. ?$ ftp 10.66.X.X Connected to 10.66.X.X 220 (vsFTPd 2.2.2) Name (10.66.74.141:zama): anonymous 331 Please specify the password. Password: 230 Login successful. Remote system type is UNIX. Using binary mode to transfer files. After that I disconnected the network cable from the server and was monitoring the status of the connection on the client side . The status of the connection was like this before and after disconnecting the network cable. --- $? for i in {1..1000} ; do netstat -at | egrep "ftp" ; date? ; sleep 60? ; done tcp??????? 0????? 0 edgebeauty.c:50179 shopfrobsoon.c:ftp ESTABLISHED Wed Feb 23 11:47:53 IST 2011 tcp??????? 0????? 0 edgebeauty.c:50179 shopfrobsoon.c:ftp ESTABLISHED Wed Feb 23 11:48:53 IST 2011 tcp??????? 0????? 0 edgebeauty.c:50179 shopfrobsoon.c:ftp ESTABLISHED Wed Feb 23 11:49:53 IST 2011 ... ... Wed Feb 23 12:14:03 IST 2011 tcp??????? 0????? 0 edgebeauty.c:50179 shopfrobsoon.c:ftp ESTABLISHED Wed Feb 23 12:15:03 IST 2011 === If we see that the time is more than 25 minutes when the server went down and the client has still maintained the connection in established state. My understanding is that the client should close the connection after TCP restarsmit timeout happens or my understanding is wrong. Please clarify . --Zaman Message: 2 Date: Tue, 22 Feb 2011 09:55:13 -0500 From: Yan Cai Subject: Re: [e2e] query on behaviour of tcp_keepalive and tcp ??? retransmit on??? Linux based systems To: end2end-interest at postel.org Message-ID: <4D63CE50.8050606 at ecs.umass.edu> Content-Type: text/plain; charset="iso-8859-1" Hi According to your description, the expected behavior should be as follows. At the beginning senders at one side can send data to the receivers at the other side, and the receivers can receive data without any problem. When some of the receivers become off-line, the affected senders should no long receive positive acknowledgments, therefore, lowering their congestion windows (i.e., sending rate). Since in your case the receiver is off forever, some senders should further experience timeout events. After a few timeouts, the sender should CLOSE this connection itself. As far as I know, the whole procedure above should be automatically invoked in the sender side. This is how TCP (sender) handles exceptions. My suggestion is that you run a simple experiment on your side to see if TCP in your machine can work that way. The test can be done using i-perf to send a long long live TCP flow, and then take off the receiver in the middle of the transmission. The connection is expected to be closed very soon after the receiver is off. Hope it helpful. Yan On 2/22/2011 4:24 AM, Zama Ques wrote: > We need some clarifications on TCP_keepalive .? We are facing some > issues on our Prod servers related to TCP functionality . > > The issue is like this. > > We have some machines at one end sending data in real time to another > group of machines on the other hand .? Now due to some hardware issues > on the other hand , some of the machines becomes unresponsive/crashes. > The client system which pumps data never came to know that the server > went unresponsive . The connection remains in > ESTABLISHED state and the client always tries to send data thinking > that the connection is alive because of which we are seeing backlog on > client sides. > > Our understanding is like this on how TCP will handle the connection. > > > Q 1) Since? the server went down , the client will try to the > retransmit the data until it times out. What is the behavior of TCP > after the timeout? Need clarification on > the following things. > a) Will the kernel will close the established connection after the > timeout . Looks like no in our case as we still see the connection > still in ESTABLISHED state after around more > than 2 hours. > b) Are there any kernel parameters which decides the when the client > is timeout after retransmission fails. What is the behavior of TCP > after the client retransmission timeouts. > > > Q 2 ) There is something called tcp_keepalive which if implemented in > the kernel , by default it's there and comes to be around 2 hrs 2 > minsutes , i think? ,? the client will send some TCP probes after the > keepalive time ineterval and if it cannot reach the server , then the > established connection in the client side will be closed by the kernel > . This is my understanding. But I can see that the connection still > remains in established after the tcp_keepalive time . We waited for > around 2 hrs 30 minutes but the connection remains in established > state only. Tried reducing the keepalive time to be around 10 minutes > , but the connection remains in ESTABLISHED state in client side . > > > Where I went wrong .Please clarify my doubts raised above . What > should we do to resolve the problem we are seeing above . Any help > will be highly appreciated as we are going through a hard time to > resolve the issue . > > Thanks in Advance > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20110222/50be8540/attachment-0001.html ------------------------------ _______________________________________________ end2end-interest mailing list end2end-interest at postel.org http://mailman.postel.org/mailman/listinfo/end2end-interest End of end2end-interest Digest, Vol 83, Issue 4 *********************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20110224/1f446983/attachment-0001.html From detlef.bosau at web.de Thu Feb 24 03:06:25 2011 From: detlef.bosau at web.de (Detlef Bosau) Date: Thu, 24 Feb 2011 12:06:25 +0100 Subject: [e2e] query on behaviour of tcp_keepalive and tcp retransmit on Linux based systems In-Reply-To: <449322.18998.qm@web137509.mail.in.yahoo.com> References: <449322.18998.qm@web137509.mail.in.yahoo.com> Message-ID: <4D663BB1.1010106@web.de> First of all, I'm not quite sure whether this is the right list for Linux specific issues. Second, you should have a look at the basic function of TCP. When a listening socket dies, the sender will continuously face timeouts. With the following consequences: 1. The sending window shrinks to 1 segment. 2. The RTO is doubled, each time a sent packet is not acknowledged in time. Whether RTO is limited or not depends on the implementation. 3. The sending socket is shut down after some period of time. (Have a look at the various timeouts in TCP.) > We need some clarifications on TCP_keepalive . We are facing some > issues on our Prod servers related to TCP functionality . > > The issue is like this. > > We have some machines at one end sending data in real time to another > group of machines on the other hand . Now due to some hardware issues > on the other hand , some of the machines becomes unresponsive/crashes. > The client system which pumps data never came to know that the server > went unresponsive . The connection remains in > ESTABLISHED state and the client always tries to send data thinking > that the connection is alive because of which we are seeing backlog on > client sides. > > Our understanding is like this on how TCP will handle the connection. > > > Q 1) Since the server went down , the client will try to the > retransmit the data until it times out. What is the behavior of TCP > after the timeout? Need clarification on > the following things. > a) Will the kernel will close the established connection after the > timeout . Looks like no in our case as we still see the connection > still in ESTABLISHED state after around more > than 2 hours. > b) Are there any kernel parameters which decides the when the client > is timeout after retransmission fails. What is the behavior of TCP > after the client retransmission timeouts. > > > Q 2 ) There is something called tcp_keepalive which if implemented in > the kernel , by default it's there and comes to be around 2 hrs 2 > minsutes , i think , the client will send some TCP probes after the > keepalive time ineterval and if it cannot reach the server , then the > established connection in the client side will be closed by the kernel > . This is my understanding. But I can see that the connection still > remains in established after the tcp_keepalive time . We waited for > around 2 hrs 30 minutes but the connection remains in established > state only. Tried reducing the keepalive time to be around 10 minutes > , but the connection remains in ESTABLISHED state in client side . > > > Where I went wrong .Please clarify my doubts raised above . What > should we do to resolve the problem we are seeing above . Any help > will be highly appreciated as we are going through a hard time to > resolve the issue . > > Thanks in Advance > > -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de ------------------------------------------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20110224/d15babaf/attachment.html From ycai at ecs.umass.edu Thu Feb 24 06:54:50 2011 From: ycai at ecs.umass.edu (Yan Cai) Date: Thu, 24 Feb 2011 09:54:50 -0500 Subject: [e2e] end2end-interest Digest, Vol 83, Issue 4 In-Reply-To: <564728.78836.qm@web137502.mail.in.yahoo.com> References: <564728.78836.qm@web137502.mail.in.yahoo.com> Message-ID: <4D66713A.3080609@ecs.umass.edu> Hi Zama, Thanks for your detailed info. Yes. the tcp stack looks good. One quick question, what kind of FTP client application is used in your test, the previous one (where the connection issue is found) or a new third party one? If it is the previous one, then your FTP client should also work well. If you still encounter that issue, there must be something else wrong. If your test is done with a new third party client, repeat the procedure with the default client you were using. Another thing you can check is the congestion window size during the period from the time the cable is disconnected to the time the ESTABLISHED connection is done. Notice that the congestion window size is hard to measure directly, but you can do this by analyzing the trace file for the period mentioned above. Use tcpdump to monitor a particular NIC port and analyze the throughput for that particular FTP flow. The amount of transmitted data over that PERIOD should be very small because the congestion window during timeout is around 1 (in terms of packets). If all the things above work normally, you should not have seen the issues observed before. Best wishes, Yan On 2/24/2011 6:06 AM, Zama Ques wrote: > Hi Yan, > > I tried testing with iperf today. > > Started server on one side and connected to the client from another > host and after sometime disconnected the cable on the server host. > > Also , reduced tcp_keepalive to 1200 sec , so timeout value should be > like 32 minutes with the other two tcp_keepalive related kernel > parameter (probes and interval) . > > > The following are my findings . > > I can see that client terminates the ESTABLISHED connection after > around 16 minutes since the server is not reachable , that is before > the TCP keepalive timeout. > > Looks to me like this minutes is somehow related to TCP retransmission > timeout which probably is determined by the following 3 parameters > which comes to be around 18 minutes. . > > $ cat /proc/sys/net/ipv4/tcp_retries1 > 3 > $ cat /proc/sys/net/ipv4/tcp_retries2 > 15 > $ cat /proc/sys/net/ipv4/tcp_fin_timeout > 60 > > > Is my assumption correct here ? > > > The following is the netstats connection flow during my experiment > > $ for i in {1..1000} ; do netstat -atn | egrep "5001" ; date ; sleep > 60 ; done > tcp 0 447432 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > tcp 0 0 10.66.X.Y:43531 > 10.66.A.B:5001 TIME_WAIT > Thu Feb 24 14:47:16 IST 2011 > tcp 0 3311576 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 14:48:16 IST 2011 > (Network Cable removed during this time from the server) > tcp 0 3317368 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 14:49:16 IST 2011 > tcp 0 3021976 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 14:50:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 14:51:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 14:52:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 14:53:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 14:54:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 14:55:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 14:56:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 14:57:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 14:58:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 14:59:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 15:00:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 15:01:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 15:02:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 15:03:16 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > Thu Feb 24 15:04:17 IST 2011 > tcp 0 2511048 10.66.X.Y:43533 > 10.66.A.B:5001 ESTABLISHED > ..(comes out be arnd 15 minutes from the server went unreachable when > the connection status changed in client) > > Thu Feb 24 15:05:17 IST 2011 > Thu Feb 24 15:06:17 IST 2011 > Thu Feb 24 15:07:17 IST 2011 > > > The following packet flow can be seen on client as sniffed by tcpdump > during while I removed the network cable . > > ===== > 14:50:20.158794 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1801285561:1801350721, > ack 0, win 92, options [nop,nop,TS val 172872553 ecr 177048109], > length 65160 > 14:50:20.164550 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1801350721:1801415881, > ack 0, win 92, options [nop,nop,TS val 172872558 ecr 177048115], > length 65160 > > 14:50:20.394916 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 172872992 ecr 177048116], > length 1448 > 14:50:21.258921 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 172873856 ecr 177048116], > length 1448 > 14:50:22.986922 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 172875584 ecr 177048116], > length 1448 > 14:50:26.442922 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 172879040 ecr 177048116], > length 1448 > 14:50:33.354923 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 172885952 ecr 177048116], > length 1448 > 14:50:47.178932 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 172899776 ecr 177048116], > length 1448 > 14:51:14.826929 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 172927424 ecr 177048116], > length 1448 > 14:52:10.122922 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 172982720 ecr 177048116], > length 1448 > 14:54:00.714934 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 173093312 ecr 177048116], > length 1448 > 14:56:00.714921 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 173213312 ecr 177048116], > length 1448 > > 14:58:00.714920 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 173333312 ecr 177048116], > length 1448 > 15:00:00.714921 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 173453312 ecr 177048116], > length 1448 > 15:02:00.714921 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 173573312 ecr 177048116], > length 1448 > 15:04:00.714936 IP edgebeauty-dr.43533 > > shopfrobsoon-dr.commplex-link: Flags [.], seq 1798969993:1798971441, > ack 0, win 92, options [nop,nop,TS val 173693312 ecr 177048116], > length 1448 > > > Does my TCP stack look fine based on the experiments above . > > > Thanks > Zaman > > > > > > > --- On *Wed, 23/2/11, Yan Cai //* wrote: > > > From: Yan Cai > Subject: Re: end2end-interest Digest, Vol 83, Issue 4 > To: "Zama Ques" > Date: Wednesday, 23 February, 2011, 2:04 PM > > Hi Zaman, > > I guess there might be some unknown ftp configuration at CLIENT > side that causes this issue. You can isolate the problem first. > I-perf can be used to test functionality of tcp stack on your > machine. If it works as expected, then there is nothing wrong with > tcp stack. Next check the settings of the ftp client (not the ftp > server) to see if there is any specific configuration that causes > this problem. If it is hard to do that, my suggestion is to > install a third party ftp client application and test with that. > > If none of them works, you might have to trace the traffic over > the cable attached to the client machine and determine what is > going on. > > Best wishes, > Yan > > On 2/23/2011 1:52 AM, Zama Ques wrote: >> Hi Yan, >> >> Thanks for your suggestion . I am familiar with iperf but the >> issue with us that it is a prod network and it is advisable for >> me not to pump data on the network . Will try to the experiment >> between two desktops connected by a cross over cable. >> >> What I was trying earlier was that I started FTP server on one >> end and connected to the server from the client side. >> >> $ ftp 10.66.X.X >> Connected to 10.66.X.X >> 220 (vsFTPd 2.2.2) >> Name (10.66.74.141:zama): anonymous >> 331 Please specify the password. >> Password: >> 230 Login successful. >> Remote system type is UNIX. >> Using binary mode to transfer files. >> >> >> After that I disconnected the network cable from the server and >> was monitoring the status of the connection on the client side . >> The status of the connection was like this before and after >> disconnecting the network cable. >> >> --- >> $ for i in {1..1000} ; do netstat -at | egrep "ftp" ; date ; >> sleep 60 ; done >> tcp 0 0 edgebeauty.c:50179 shopfrobsoon.c:ftp >> ESTABLISHED >> Wed Feb 23 11:47:53 IST 2011 >> >> tcp 0 0 edgebeauty.c:50179 shopfrobsoon.c:ftp >> ESTABLISHED >> Wed Feb 23 11:48:53 IST 2011 >> tcp 0 0 edgebeauty.c:50179 shopfrobsoon.c:ftp >> ESTABLISHED >> Wed Feb 23 11:49:53 IST 2011 >> ... >> ... >> Wed Feb 23 12:14:03 IST 2011 >> tcp 0 0 edgebeauty.c:50179 shopfrobsoon.c:ftp >> ESTABLISHED >> Wed Feb 23 12:15:03 IST 2011 >> === >> >> If we see that the time is more than 25 minutes when the server >> went down and the client has still maintained the connection in >> established state. >> >> My understanding is that the client should close the connection >> after TCP restarsmit timeout happens or my understanding is wrong. >> >> Please clarify . >> >> --Zaman >> >> >> Message: 2 >> Date: Tue, 22 Feb 2011 09:55:13 -0500 >> From: Yan Cai >> Subject: Re: [e2e] query on behaviour of tcp_keepalive and tcp >> retransmit on Linux based systems >> To: end2end-interest at postel.org >> Message-ID: <4D63CE50.8050606 at ecs.umass.edu> >> Content-Type: text/plain; charset="iso-8859-1" >> >> Hi >> >> According to your description, the expected behavior should >> be as follows. >> At the beginning senders at one side can send data to the >> receivers at >> the other side, and the receivers can receive data without >> any problem. >> When some of the receivers become off-line, the affected >> senders should >> no long receive positive acknowledgments, therefore, lowering >> their >> congestion windows (i.e., sending rate). Since in your case >> the receiver >> is off forever, some senders should further experience >> timeout events. >> After a few timeouts, the sender should CLOSE this connection >> itself. >> >> As far as I know, the whole procedure above should be >> automatically >> invoked in the sender side. This is how TCP (sender) handles >> exceptions. >> >> My suggestion is that you run a simple experiment on your >> side to see if >> TCP in your machine can work that way. The test can be done >> using i-perf >> to send a long long live TCP flow, and then take off the >> receiver in the >> middle of the transmission. The connection is expected to be >> closed very >> soon after the receiver is off. >> >> Hope it helpful. >> Yan >> On 2/22/2011 4:24 AM, Zama Ques wrote: >> > We need some clarifications on TCP_keepalive . We are >> facing some >> > issues on our Prod servers related to TCP functionality . >> > >> > The issue is like this. >> > >> > We have some machines at one end sending data in real time >> to another >> > group of machines on the other hand . Now due to some >> hardware issues >> > on the other hand , some of the machines becomes >> unresponsive/crashes. >> > The client system which pumps data never came to know that >> the server >> > went unresponsive . The connection remains in >> > ESTABLISHED state and the client always tries to send data >> thinking >> > that the connection is alive because of which we are seeing >> backlog on >> > client sides. >> > >> > Our understanding is like this on how TCP will handle the >> connection. >> > >> > >> > Q 1) Since the server went down , the client will try to the >> > retransmit the data until it times out. What is the >> behavior of TCP >> > after the timeout? Need clarification on >> > the following things. >> > a) Will the kernel will close the established connection >> after the >> > timeout . Looks like no in our case as we still see the >> connection >> > still in ESTABLISHED state after around more >> > than 2 hours. >> > b) Are there any kernel parameters which decides the when >> the client >> > is timeout after retransmission fails. What is the behavior >> of TCP >> > after the client retransmission timeouts. >> > >> > >> > Q 2 ) There is something called tcp_keepalive which if >> implemented in >> > the kernel , by default it's there and comes to be around 2 >> hrs 2 >> > minsutes , i think , the client will send some TCP probes >> after the >> > keepalive time ineterval and if it cannot reach the server >> , then the >> > established connection in the client side will be closed by >> the kernel >> > . This is my understanding. But I can see that the >> connection still >> > remains in established after the tcp_keepalive time . We >> waited for >> > around 2 hrs 30 minutes but the connection remains in >> established >> > state only. Tried reducing the keepalive time to be around >> 10 minutes >> > , but the connection remains in ESTABLISHED state in client >> side . >> > >> > >> > Where I went wrong .Please clarify my doubts raised above . >> What >> > should we do to resolve the problem we are seeing above . >> Any help >> > will be highly appreciated as we are going through a hard >> time to >> > resolve the issue . >> > >> > Thanks in Advance >> > >> > >> >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL: >> http://mailman.postel.org/pipermail/end2end-interest/attachments/20110222/50be8540/attachment-0001.html >> >> ------------------------------ >> >> _______________________________________________ >> end2end-interest mailing list >> end2end-interest at postel.org >> http://mailman.postel.org/mailman/listinfo/end2end-interest >> >> >> End of end2end-interest Digest, Vol 83, Issue 4 >> *********************************************** >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20110224/5e36eca8/attachment-0001.html