From detlef.bosau at web.de Sun Dec 1 07:57:28 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sun, 01 Dec 2013 16:57:28 +0100 Subject: [e2e] Answer to Dave Reed Re: Fwd: Re: Question the other way round: In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> Message-ID: <529B5C68.9060202@web.de> Am 01.12.2013 06:05, schrieb Andrew Mcgregor: > The actual clearance rate from the queue (or the sojourn time), if that > matters for your AQM scheme. That way you are not assuming a known line > rate. Clearance rate or sojourn time? Clearance rate may apply for a packet delivery rate. From a TCP point of view, the sojourn time is the difference between the arrival of the according ACK and the time a data packet left the sender. So you omit any recovery latency. > > On 30 November 2013 00:13, Detlef Bosau wrote: > >> Am 29.11.2013 00:24, schrieb Andrew Mcgregor: >> >> In which case... measure, don't assume. Served us well for 802.11 >> modulation selection, I don't see why it shouldn't work for AQM. >> >> >> What do you want to measure? >> > > -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From andrewmcgr at google.com Sun Dec 1 13:16:08 2013 From: andrewmcgr at google.com (Andrew Mcgregor) Date: Mon, 2 Dec 2013 08:16:08 +1100 Subject: [e2e] Answer to Dave Reed Re: Fwd: Re: Question the other way round: In-Reply-To: <529B5C68.9060202@web.de> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> Message-ID: I mean sojourn time, one way in the particular queue, as per CoDel, rather than anything TCP-related. Clearance rate is fairly simply related to sojourn time, of course, given enough integration time for the statistics to converge. On 2 December 2013 02:57, Detlef Bosau wrote: > Am 01.12.2013 06:05, schrieb Andrew Mcgregor: > > The actual clearance rate from the queue (or the sojourn time), if that > > matters for your AQM scheme. That way you are not assuming a known line > > rate. > Clearance rate or sojourn time? > > Clearance rate may apply for a packet delivery rate. From a TCP point of > view, the sojourn time is the difference between the arrival of the > according ACK and the time a data packet left the sender. > > So you omit any recovery latency. > > > > > > > On 30 November 2013 00:13, Detlef Bosau wrote: > > > >> Am 29.11.2013 00:24, schrieb Andrew Mcgregor: > >> > >> In which case... measure, don't assume. Served us well for 802.11 > >> modulation selection, I don't see why it shouldn't work for AQM. > >> > >> > >> What do you want to measure? > >> > > > > > > > -- > ------------------------------------------------------------------ > Detlef Bosau > Galileistra?e 30 > 70565 Stuttgart Tel.: +49 711 5208031 > mobile: +49 172 6819937 > skype: detlef.bosau > ICQ: 566129673 > detlef.bosau at web.de http://www.detlef-bosau.de > > -- Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 From detlef.bosau at web.de Sun Dec 1 13:23:17 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sun, 01 Dec 2013 22:23:17 +0100 Subject: [e2e] Answer to Dave Reed Re: Fwd: Re: Question the other way round: In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> Message-ID: <529BA8C5.4050500@web.de> Am 01.12.2013 22:16, schrieb Andrew Mcgregor: > I mean sojourn time, one way in the particular queue, as per CoDel, > rather than anything TCP-related. Clearance rate is fairly simply > related to sojourn time, of course, given enough integration time for > the statistics to converge. What will "converge"? Either we talk completely at cross purposes or our views are incompatible. Particularly: As long as we talk about IP, we have to talk about clearance times for IP packets - and the observed packet corruption rate should be negligible. > > > On 2 December 2013 02:57, Detlef Bosau > wrote: > > Am 01.12.2013 06:05, schrieb Andrew Mcgregor: > > The actual clearance rate from the queue (or the sojourn time), > if that > > matters for your AQM scheme. That way you are not assuming a > known line > > rate. > Clearance rate or sojourn time? > > Clearance rate may apply for a packet delivery rate. From a TCP > point of > view, the sojourn time is the difference between the arrival of the > according ACK and the time a data packet left the sender. > > So you omit any recovery latency. > > > > > > > On 30 November 2013 00:13, Detlef Bosau > wrote: > > > >> Am 29.11.2013 00:24, schrieb Andrew Mcgregor: > >> > >> In which case... measure, don't assume. Served us well for 802.11 > >> modulation selection, I don't see why it shouldn't work for AQM. > >> > >> > >> What do you want to measure? > >> > > > > > > > -- > ------------------------------------------------------------------ > Detlef Bosau > Galileistra?e 30 > 70565 Stuttgart Tel.: +49 711 5208031 > > mobile: +49 172 6819937 > > skype: detlef.bosau > ICQ: 566129673 > detlef.bosau at web.de > http://www.detlef-bosau.de > > > > > -- > Andrew McGregor | SRE | andrewmcgr at google.com > | +61 4 8143 7128 -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From detlef.bosau at web.de Tue Dec 3 11:11:55 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Tue, 03 Dec 2013 20:11:55 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> Message-ID: <529E2CFB.9050307@web.de> To my understanding, the "sojourn time" considered in CoDel is the difference between the time when a packet/leaves /a queue and the time, when this packet has /arrived /at the queue. In other words: The time a packet spends in the queue. When this time is unusually high, CoDel sees an imminent congestion and drops packets. The problem is that CoDel makes no difference whether the "sojourn time" is caused by a huge number of packets in the queue, i.e. congestion, or by a huge delivery time resulting from corruption loss and necessary retransmissions. Hence, we have the good old loss differentiation problem. And because CoDel is particularly intended for edge routers, the disaster is placed exactly there where it is expected to happen......8-) Detlef Am 01.12.2013 22:16, schrieb Andrew Mcgregor: > I mean sojourn time, one way in the particular queue, as per CoDel, rather > than anything TCP-related. Clearance rate is fairly simply related to > sojourn time, of course, given enough integration time for the statistics > to converge. > > > On 2 December 2013 02:57, Detlef Bosau wrote: > >> Am 01.12.2013 06:05, schrieb Andrew Mcgregor: >>> The actual clearance rate from the queue (or the sojourn time), if that >>> matters for your AQM scheme. That way you are not assuming a known line >>> rate. >> Clearance rate or sojourn time? >> >> Clearance rate may apply for a packet delivery rate. From a TCP point of >> view, the sojourn time is the difference between the arrival of the >> according ACK and the time a data packet left the sender. >> >> So you omit any recovery latency. >> >> >> >>> On 30 November 2013 00:13, Detlef Bosau wrote: >>> >>>> Am 29.11.2013 00:24, schrieb Andrew Mcgregor: >>>> >>>> In which case... measure, don't assume. Served us well for 802.11 >>>> modulation selection, I don't see why it shouldn't work for AQM. >>>> >>>> >>>> What do you want to measure? >>>> >>> >> >> -- >> ------------------------------------------------------------------ >> Detlef Bosau >> Galileistra?e 30 >> 70565 Stuttgart Tel.: +49 711 5208031 >> mobile: +49 172 6819937 >> skype: detlef.bosau >> ICQ: 566129673 >> detlef.bosau at web.de http://www.detlef-bosau.de >> >> > -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From dhavey at yahoo.com Tue Dec 3 13:11:19 2013 From: dhavey at yahoo.com (Daniel Havey) Date: Tue, 3 Dec 2013 13:11:19 -0800 (PST) Subject: [e2e] Codel and Wireless In-Reply-To: <529E2CFB.9050307@web.de> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> Message-ID: <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> On Tuesday, December 3, 2013 12:22 PM, Detlef Bosau wrote: To my understanding, the "sojourn time" considered in CoDel is the difference between the time when a packet/leaves /a queue and the time, when this packet has /arrived /at the queue. In other words: The time a packet spends in the queue. When this time is unusually high, CoDel sees an imminent congestion and drops packets. The problem is that CoDel makes no difference whether the "sojourn time" is caused by a huge number of packets in the queue, i.e. congestion, or by a huge delivery time resulting from corruption loss and necessary retransmissions. CoDel parameters are interval and target.? If the queue drains before the interval then there shouldn't be any drops.?? Also there is "leverage" from Red in a Different Light.? If CoDel decides to drop a packet from a flow that is in congestion avoidance, fast retransmit or slow start then the window is halved and the queue drains quickly.? If the flow doesn't have enough data to trigger fast retransmit then that is unfortunate for that user since they now have to wait an RTT for that packet, and it does not drain the queue very much. Hence, we have the good old loss differentiation problem. And because CoDel is particularly intended for edge routers, the disaster is placed exactly there where it is expected to happen......8-) Detlef Am 01.12.2013 22:16, schrieb Andrew Mcgregor: > I mean sojourn time, one way in the particular queue, as per CoDel, rather > than anything TCP-related.? Clearance rate is fairly simply related to > sojourn time, of course, given enough integration time for the statistics > to converge. > > > On 2 December 2013 02:57, Detlef Bosau wrote: > >> Am 01.12.2013 06:05, schrieb Andrew Mcgregor: >>> The actual clearance rate from the queue (or the sojourn time), if that >>> matters for your AQM scheme.? That way you are not assuming a known line >>> rate. >> Clearance rate or sojourn time? >> >> Clearance rate may apply for a packet delivery rate. From a TCP point of >> view, the sojourn time is the difference between the arrival of the >> according ACK and the time a data packet left the sender. >> >> So you omit any recovery latency. >> >> >> >>> On 30 November 2013 00:13, Detlef Bosau wrote: >>> >>>>? Am 29.11.2013 00:24, schrieb Andrew Mcgregor: >>>> >>>> In which case... measure, don't assume.? Served us well for 802.11 >>>> modulation selection, I don't see why it shouldn't work for AQM. >>>> >>>> >>>> What do you want to measure? >>>> >>> >> >> -- >> ------------------------------------------------------------------ >> Detlef Bosau >> Galileistra?e 30 >> 70565 Stuttgart? ? ? ? ? ? ? ? ? ? ? ? ? ? Tel.:? +49 711 5208031 >>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? mobile: +49 172 6819937 >>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? skype:? ? detlef.bosau >>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ICQ:? ? ? ? ? 566129673 >> detlef.bosau at web.de? ? ? ? ? ? ? ? ? ? http://www.detlef-bosau.de >> >> > -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30? 70565 Stuttgart? ? ? ? ? ? ? ? ? ? ? ? ? ? Tel.:? +49 711 5208031 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? mobile: +49 172 6819937 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? skype:? ? detlef.bosau ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ICQ:? ? ? ? ? 566129673 detlef.bosau at web.de? ? ? ? ? ? ? ? ? ? http://www.detlef-bosau.de From andrewmcgr at google.com Tue Dec 3 14:41:29 2013 From: andrewmcgr at google.com (Andrew Mcgregor) Date: Wed, 4 Dec 2013 09:41:29 +1100 Subject: [e2e] Codel and Wireless In-Reply-To: <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> Message-ID: All of which is why fq_codel is so much better... because flows queue independently, and drops are calculated per flow (although overall queue size is included implicitly via the sojourn time), the RTT delay has far less impact. CoDel is an ingredient of an AQM system, not a desirable AQM on its own. On 4 December 2013 08:11, Daniel Havey wrote: > > > > > > On Tuesday, December 3, 2013 12:22 PM, Detlef Bosau > wrote: > > To my understanding, the "sojourn time" considered in CoDel is the > difference between the time when a packet/leaves /a queue and the time, > when this packet has /arrived /at the queue. In other words: The time a > packet spends in the queue. > > When this time is unusually high, CoDel sees an imminent congestion and > drops packets. > > The problem is that CoDel makes no difference whether the "sojourn time" > is caused by a huge number of packets in the queue, i.e. congestion, or > by a huge delivery time resulting from corruption loss and necessary > retransmissions. > > CoDel parameters are interval and target. If the queue drains before the > interval then there shouldn't be any drops. Also there is "leverage" from > Red in a Different Light. If CoDel decides to drop a packet from a flow > that is in congestion avoidance, fast retransmit or slow start then the > window is halved and the queue drains quickly. If the flow doesn't have > enough data to trigger fast retransmit then that is unfortunate for that > user since they now have to wait an RTT for that packet, and it does not > drain the queue very much. > > > Hence, we have the good old loss differentiation problem. And because > CoDel is particularly intended for edge routers, the disaster is placed > exactly there where it is expected to happen......8-) > > Detlef > > Am 01.12.2013 22:16, schrieb Andrew Mcgregor: > > I mean sojourn time, one way in the particular queue, as per CoDel, > rather > > than anything TCP-related. Clearance rate is fairly simply related to > > sojourn time, of course, given enough integration time for the statistics > > to converge. > > > > > > On 2 December 2013 02:57, Detlef Bosau wrote: > > > >> Am 01.12.2013 06:05, schrieb Andrew Mcgregor: > >>> The actual clearance rate from the queue (or the sojourn time), if that > >>> matters for your AQM scheme. That way you are not assuming a known > line > >>> rate. > >> Clearance rate or sojourn time? > >> > >> Clearance rate may apply for a packet delivery rate. From a TCP point of > >> view, the sojourn time is the difference between the arrival of the > >> according ACK and the time a data packet left the sender. > >> > >> So you omit any recovery latency. > >> > >> > >> > >>> On 30 November 2013 00:13, Detlef Bosau wrote: > >>> > >>>> Am 29.11.2013 00:24, schrieb Andrew Mcgregor: > >>>> > >>>> In which case... measure, don't assume. Served us well for 802.11 > >>>> modulation selection, I don't see why it shouldn't work for AQM. > >>>> > >>>> > >>>> What do you want to measure? > >>>> > >>> > >> > >> -- > >> ------------------------------------------------------------------ > >> Detlef Bosau > >> Galileistra?e 30 > >> 70565 Stuttgart Tel.: +49 711 5208031 > >> mobile: +49 172 6819937 > >> skype: detlef.bosau > >> ICQ: 566129673 > >> detlef.bosau at web.de http://www.detlef-bosau.de > > >> > >> > > > > > -- > ------------------------------------------------------------------ > Detlef Bosau > Galileistra?e 30 > 70565 Stuttgart Tel.: +49 711 5208031 > mobile: +49 172 6819937 > skype: detlef.bosau > ICQ: 566129673 > detlef.bosau at web.de http://www.detlef-bosau.de > -- Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 From dhavey at yahoo.com Tue Dec 3 15:10:41 2013 From: dhavey at yahoo.com (Daniel Havey) Date: Tue, 3 Dec 2013 15:10:41 -0800 (PST) Subject: [e2e] Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> Message-ID: <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> On Tuesday, December 3, 2013 2:41 PM, Andrew Mcgregor wrote: All of which is why fq_codel is so much better... because flows queue independently, and drops are calculated per flow (although overall queue size is included implicitly via the sojourn time), the RTT delay has far less impact. ?CoDel is an ingredient of an AQM system, not a desirable AQM on its own. >Makes sense to me. ?We need to get the worst case RTT right. ?If we set the interval to 100ms then the user with the users with larger RTTs may have issues. On 4 December 2013 08:11, Daniel Havey wrote: > > > > >On Tuesday, December 3, 2013 12:22 PM, Detlef Bosau wrote: > >To my understanding, the "sojourn time" considered in CoDel is the >difference between the time when a packet/leaves /a queue and the time, >when this packet has /arrived /at the queue. In other words: The time a >packet spends in the queue. > >When this time is unusually high, CoDel sees an imminent congestion and >drops packets. > >The problem is that CoDel makes no difference whether the "sojourn time" >is caused by a huge number of packets in the queue, i.e. congestion, or >by a huge delivery time resulting from corruption loss and necessary >retransmissions. > >CoDel parameters are interval and target.? If the queue drains before the interval then there shouldn't be any drops.?? Also there is "leverage" from Red in a Different Light.? If CoDel decides to drop a packet from a flow that is in congestion avoidance, fast retransmit or slow start then the window is halved and the queue drains quickly.? If the flow doesn't have enough data to trigger fast retransmit then that is unfortunate for that user since they now have to wait an RTT for that packet, and it does not drain the queue very much. > > > >Hence, we have the good old loss differentiation problem. And because >CoDel is particularly intended for edge routers, the disaster is placed >exactly there where it is expected to happen......8-) > >Detlef > >Am 01.12.2013 22:16, schrieb Andrew Mcgregor: >> I mean sojourn time, one way in the particular queue, as per CoDel, rather >> than anything TCP-related.? Clearance rate is fairly simply related to >> sojourn time, of course, given enough integration time for the statistics >> to converge. >> >> >> On 2 December 2013 02:57, Detlef Bosau wrote: >> >>> Am 01.12.2013 06:05, schrieb Andrew Mcgregor: >>>> The actual clearance rate from the queue (or the sojourn time), if that >>>> matters for your AQM scheme.? That way you are not assuming a known line >>>> rate. >>> Clearance rate or sojourn time? >>> >>> Clearance rate may apply for a packet delivery rate. From a TCP point of >>> view, the sojourn time is the difference between the arrival of the >>> according ACK and the time a data packet left the sender. >>> >>> So you omit any recovery latency. >>> >>> >>> >>>> On 30 November 2013 00:13, Detlef Bosau wrote: >>>> >>>>>? Am 29.11.2013 00:24, schrieb Andrew Mcgregor: >>>>> >>>>> In which case... measure, don't assume.? Served us well for 802.11 >>>>> modulation selection, I don't see why it shouldn't work for AQM. >>>>> >>>>> >>>>> What do you want to measure? >>>>> >>>> >>> >>> -- >>> ------------------------------------------------------------------ >>> Detlef Bosau >>> Galileistra?e 30 >>> 70565 Stuttgart? ? ? ? ? ? ? ? ? ? ? ? ? ? Tel.:? ?+49 711 5208031 >>>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? mobile: +49 172 6819937 >>>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? skype:? ? ?detlef.bosau >>>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ICQ:? ? ? ? ? 566129673 >>> detlef.bosau at web.de? ? ? ? ? ? ? ? ? ? http://www.detlef-bosau.de > >>> >>> >> > > >-- >------------------------------------------------------------------ >Detlef Bosau >Galileistra?e 30? >70565 Stuttgart? ? ? ? ? ? ? ? ? ? ? ? ? ? Tel.:? ?+49 711 5208031 >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mobile: +49 172 6819937 >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?skype:? ? ?detlef.bosau >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ICQ:? ? ? ? ? 566129673 >detlef.bosau at web.de? ? ? ? ? ? ? ? ? ? http://www.detlef-bosau.de > -- Andrew McGregor?|?SRE?|?andrewmcgr at google.com?|?+61 4 8143 7128 From andrewmcgr at google.com Tue Dec 3 15:45:58 2013 From: andrewmcgr at google.com (Andrew Mcgregor) Date: Wed, 4 Dec 2013 10:45:58 +1100 Subject: [e2e] Codel and Wireless In-Reply-To: <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> Message-ID: Empirically, for fq_codel, long RTT flows work fine so long as RTT < 5 intervals, roughly speaking, and it degrades very slowly. So 100ms is about right for the internet. On 4 December 2013 10:10, Daniel Havey wrote: > On Tuesday, December 3, 2013 2:41 PM, Andrew Mcgregor < > andrewmcgr at google.com> wrote: > All of which is why fq_codel is so much better... because flows queue > independently, and drops are calculated per flow (although overall queue > size is included implicitly via the sojourn time), the RTT delay has far > less impact. CoDel is an ingredient of an AQM system, not a desirable AQM > on its own. > > >Makes sense to me. We need to get the worst case RTT right. If we set > the interval to 100ms then the user with the users with larger RTTs may > have issues. > > > On 4 December 2013 08:11, Daniel Havey wrote: > > > > > > > On Tuesday, December 3, 2013 12:22 PM, Detlef Bosau > wrote: > > To my understanding, the "sojourn time" considered in CoDel is the > difference between the time when a packet/leaves /a queue and the time, > when this packet has /arrived /at the queue. In other words: The time a > packet spends in the queue. > > When this time is unusually high, CoDel sees an imminent congestion and > drops packets. > > The problem is that CoDel makes no difference whether the "sojourn time" > is caused by a huge number of packets in the queue, i.e. congestion, or > by a huge delivery time resulting from corruption loss and necessary > retransmissions. > > CoDel parameters are interval and target. If the queue drains before the > interval then there shouldn't be any drops. Also there is "leverage" from > Red in a Different Light. If CoDel decides to drop a packet from a flow > that is in congestion avoidance, fast retransmit or slow start then the > window is halved and the queue drains quickly. If the flow doesn't have > enough data to trigger fast retransmit then that is unfortunate for that > user since they now have to wait an RTT for that packet, and it does not > drain the queue very much. > > > Hence, we have the good old loss differentiation problem. And because > CoDel is particularly intended for edge routers, the disaster is placed > exactly there where it is expected to happen......8-) > > Detlef > > Am 01.12.2013 22:16, schrieb Andrew Mcgregor: > > I mean sojourn time, one way in the particular queue, as per CoDel, > rather > > than anything TCP-related. Clearance rate is fairly simply related to > > sojourn time, of course, given enough integration time for the statistics > > to converge. > > > > > > On 2 December 2013 02:57, Detlef Bosau wrote: > > > >> Am 01.12.2013 06:05, schrieb Andrew Mcgregor: > >>> The actual clearance rate from the queue (or the sojourn time), if that > >>> matters for your AQM scheme. That way you are not assuming a known > line > >>> rate. > >> Clearance rate or sojourn time? > >> > >> Clearance rate may apply for a packet delivery rate. From a TCP point of > >> view, the sojourn time is the difference between the arrival of the > >> according ACK and the time a data packet left the sender. > >> > >> So you omit any recovery latency. > >> > >> > >> > >>> On 30 November 2013 00:13, Detlef Bosau wrote: > >>> > >>>> Am 29.11.2013 00:24, schrieb Andrew Mcgregor: > >>>> > >>>> In which case... measure, don't assume. Served us well for 802.11 > >>>> modulation selection, I don't see why it shouldn't work for AQM. > >>>> > >>>> > >>>> What do you want to measure? > >>>> > >>> > >> > >> -- > >> ------------------------------------------------------------------ > >> Detlef Bosau > >> Galileistra?e 30 > >> 70565 Stuttgart Tel.: +49 711 5208031 > >> mobile: +49 172 6819937 > >> skype: detlef.bosau > >> ICQ: 566129673 > >> detlef.bosau at web.de http://www.detlef-bosau.de > > >> > >> > > > > > -- > ------------------------------------------------------------------ > Detlef Bosau > Galileistra?e 30 > 70565 Stuttgart Tel.: +49 711 5208031 > mobile: +49 172 6819937 > skype: detlef.bosau > ICQ: 566129673 > detlef.bosau at web.de http://www.detlef-bosau.de > > > > > -- > Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 > > > -- Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 From keithw at mit.edu Tue Dec 3 16:54:15 2013 From: keithw at mit.edu (Keith Winstein) Date: Tue, 3 Dec 2013 19:54:15 -0500 Subject: [e2e] Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> Message-ID: Folks here might be interested -- my colleague Anirudh Sivaraman has done some quantitative testing on sfqCoDel (the ns-2 module) vs. CoDel (one queue) vs. large DropTail queues (with per-flow queueing), in ns-2 emulation of a Verizon LTE link. Paper, code and replication instructions here: http://web.mit.edu/anirudh/www/sdn-data-plane.html We emulate a commercial LTE link queue, where datagrams arrive at the back whenever somebody chooses to send one, and are serviced (and delivered) at particular instants if the queue is nonempty. We recorded those instants from a real LTE network, by keeping the queue nonempty and then measuring when datagrams were delivered to the other side, and then play back the same services in emulation. (The same strategy was used in our Sprout work.) The summary is that we find fq_codel is pretty reasonable overall, but there are cases where it suffers considerably on some metrics that may be worth knowing about. For instance, the replication dataset includes a case where a bulk flow of TCP NewReno gets 12 Mbps over LTE (with a bufferbloated link), and only 4.4 Mbps if the LTE link is controlled with fq_codel or codel. There's also a case where we look at a contending bulk-throughput application and a Web-like application going over the same link, and *both* workloads end up doing a lot better over the bufferbloated, per-flow-queue LTE link than the fq_codel link. This is with a single real-world trace that was recorded from Verizon's LTE network in Boston. I don't think our dataset is sufficient to start making claims about whether fq_codel is or isn't good in the real world and whether Verizon should implement it on their base station queues (and whether Qualcomm should implement it on their baseband LTE modem queues for the uplink), but it could be useful as a tool to measure the kinds of properties Andrew is talking about without having to implement CoDel on an eNodeB. AFAIK we haven't explored sweeping the CoDel parameters (the 100ms and the 5ms) to see if different values perform better empirically on these traces, but maybe we should... Cheers, Keith On Tue, Dec 3, 2013 at 6:45 PM, Andrew Mcgregor wrote: > Empirically, for fq_codel, long RTT flows work fine so long as RTT < 5 > intervals, roughly speaking, and it degrades very slowly. So 100ms is > about right for the internet. > > > On 4 December 2013 10:10, Daniel Havey wrote: > > > On Tuesday, December 3, 2013 2:41 PM, Andrew Mcgregor < > > andrewmcgr at google.com> wrote: > > All of which is why fq_codel is so much better... because flows queue > > independently, and drops are calculated per flow (although overall queue > > size is included implicitly via the sojourn time), the RTT delay has far > > less impact. CoDel is an ingredient of an AQM system, not a desirable > AQM > > on its own. > > > > >Makes sense to me. We need to get the worst case RTT right. If we set > > the interval to 100ms then the user with the users with larger RTTs may > > have issues. > > > > > > On 4 December 2013 08:11, Daniel Havey wrote: > > > > > > > > > > > > > > On Tuesday, December 3, 2013 12:22 PM, Detlef Bosau > > > wrote: > > > > To my understanding, the "sojourn time" considered in CoDel is the > > difference between the time when a packet/leaves /a queue and the time, > > when this packet has /arrived /at the queue. In other words: The time a > > packet spends in the queue. > > > > When this time is unusually high, CoDel sees an imminent congestion and > > drops packets. > > > > The problem is that CoDel makes no difference whether the "sojourn time" > > is caused by a huge number of packets in the queue, i.e. congestion, or > > by a huge delivery time resulting from corruption loss and necessary > > retransmissions. > > > > CoDel parameters are interval and target. If the queue drains before the > > interval then there shouldn't be any drops. Also there is "leverage" > from > > Red in a Different Light. If CoDel decides to drop a packet from a flow > > that is in congestion avoidance, fast retransmit or slow start then the > > window is halved and the queue drains quickly. If the flow doesn't have > > enough data to trigger fast retransmit then that is unfortunate for that > > user since they now have to wait an RTT for that packet, and it does not > > drain the queue very much. > > > > > > Hence, we have the good old loss differentiation problem. And because > > CoDel is particularly intended for edge routers, the disaster is placed > > exactly there where it is expected to happen......8-) > > > > Detlef > > > > Am 01.12.2013 22:16, schrieb Andrew Mcgregor: > > > I mean sojourn time, one way in the particular queue, as per CoDel, > > rather > > > than anything TCP-related. Clearance rate is fairly simply related to > > > sojourn time, of course, given enough integration time for the > statistics > > > to converge. > > > > > > > > > On 2 December 2013 02:57, Detlef Bosau wrote: > > > > > >> Am 01.12.2013 06:05, schrieb Andrew Mcgregor: > > >>> The actual clearance rate from the queue (or the sojourn time), if > that > > >>> matters for your AQM scheme. That way you are not assuming a known > > line > > >>> rate. > > >> Clearance rate or sojourn time? > > >> > > >> Clearance rate may apply for a packet delivery rate. From a TCP point > of > > >> view, the sojourn time is the difference between the arrival of the > > >> according ACK and the time a data packet left the sender. > > >> > > >> So you omit any recovery latency. > > >> > > >> > > >> > > >>> On 30 November 2013 00:13, Detlef Bosau wrote: > > >>> > > >>>> Am 29.11.2013 00:24, schrieb Andrew Mcgregor: > > >>>> > > >>>> In which case... measure, don't assume. Served us well for 802.11 > > >>>> modulation selection, I don't see why it shouldn't work for AQM. > > >>>> > > >>>> > > >>>> What do you want to measure? > > >>>> > > >>> > > >> > > >> -- > > >> ------------------------------------------------------------------ > > >> Detlef Bosau > > >> Galileistra?e 30 > > >> 70565 Stuttgart Tel.: +49 711 5208031 > > >> mobile: +49 172 6819937 > > >> skype: detlef.bosau > > >> ICQ: 566129673 > > >> detlef.bosau at web.de http://www.detlef-bosau.de > > > > >> > > >> > > > > > > > > > -- > > ------------------------------------------------------------------ > > Detlef Bosau > > Galileistra?e 30 > > 70565 Stuttgart Tel.: +49 711 5208031 > > mobile: +49 172 6819937 > > skype: detlef.bosau > > ICQ: 566129673 > > detlef.bosau at web.de http://www.detlef-bosau.de > > > > > > > > > > -- > > Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 > > > > > > > > > -- > Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 > From dhavey at yahoo.com Tue Dec 3 17:34:40 2013 From: dhavey at yahoo.com (Daniel Havey) Date: Tue, 3 Dec 2013 17:34:40 -0800 (PST) Subject: [e2e] Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> Message-ID: <1386120880.45379.YahooMailNeo@web141604.mail.bf1.yahoo.com> On Tuesday, December 3, 2013 4:54 PM, Keith Winstein wrote: Folks here might be interested -- my colleague Anirudh Sivaraman has done some quantitative testing on sfqCoDel (the ns-2 module) vs. CoDel (one queue) vs. large DropTail queues (with per-flow queueing), in ns-2 emulation of a Verizon LTE link. Paper, code and replication instructions here:?http://web.mit.edu/anirudh/www/sdn-data-plane.html We emulate a commercial LTE link queue, where datagrams arrive at the back whenever somebody chooses to send one, and are serviced (and delivered) at particular instants if the queue is nonempty. We recorded those instants from a real LTE network, by keeping the queue nonempty and then measuring when datagrams were delivered to the other side, and then play back the same services in emulation. (The same strategy was used in our Sprout work.) The summary is that we find fq_codel is pretty reasonable overall, but there are cases where it suffers considerably on some metrics that may be worth knowing about. For instance, the replication dataset includes a case where a bulk flow of TCP NewReno gets 12 Mbps over LTE (with a bufferbloated link), and only 4.4 Mbps if the LTE link is controlled with fq_codel or codel. There's also a case where we look at a contending bulk-throughput application and a Web-like application going over the same link, and *both* workloads end up doing a lot better over the bufferbloated, per-flow-queue LTE link than the fq_codel link. This is with a single real-world trace that was recorded from Verizon's LTE network in Boston. I don't think our dataset is sufficient to start making claims about whether fq_codel is or isn't good in the real world and whether Verizon should implement it on their base station queues (and whether Qualcomm should implement it on their baseband LTE modem queues for the uplink), but it could be useful as a tool to measure the kinds of properties Andrew is talking about without having to implement CoDel on an eNodeB. AFAIK we haven't explored sweeping the CoDel parameters (the 100ms and the 5ms) to see if different values perform better empirically on these traces, but maybe we should... Cheers, Keith >>> Nice study! ?I was just wondering what would happen if the CoDel parameters were not true. ?Specifically something like this: What happens to a connection that exceeds the 100ms RTT?| ...Daniel On Tue, Dec 3, 2013 at 6:45 PM, Andrew Mcgregor wrote: Empirically, for fq_codel, long RTT flows work fine so long as RTT < 5 >intervals, roughly speaking, and it degrades very slowly. ?So 100ms is >about right for the internet. > > > >On 4 December 2013 10:10, Daniel Havey wrote: > >> On Tuesday, December 3, 2013 2:41 PM, Andrew Mcgregor < >> andrewmcgr at google.com> wrote: >> ?All of which is why fq_codel is so much better... because flows queue >> independently, and drops are calculated per flow (although overall queue >> size is included implicitly via the sojourn time), the RTT delay has far >> less impact. ?CoDel is an ingredient of an AQM system, not a desirable AQM >> on its own. >> >> >Makes sense to me. ?We need to get the worst case RTT right. ?If we set >> the interval to 100ms then the user with the users with larger RTTs may >> have issues. >> >> >> On 4 December 2013 08:11, Daniel Havey wrote: >> >> >> >> >> >> >> On Tuesday, December 3, 2013 12:22 PM, Detlef Bosau >> wrote: >> >> To my understanding, the "sojourn time" considered in CoDel is the >> difference between the time when a packet/leaves /a queue and the time, >> when this packet has /arrived /at the queue. In other words: The time a >> packet spends in the queue. >> >> When this time is unusually high, CoDel sees an imminent congestion and >> drops packets. >> >> The problem is that CoDel makes no difference whether the "sojourn time" >> is caused by a huge number of packets in the queue, i.e. congestion, or >> by a huge delivery time resulting from corruption loss and necessary >> retransmissions. >> >> CoDel parameters are interval and target. ?If the queue drains before the >> interval then there shouldn't be any drops. ? Also there is "leverage" from >> Red in a Different Light. ?If CoDel decides to drop a packet from a flow >> that is in congestion avoidance, fast retransmit or slow start then the >> window is halved and the queue drains quickly. ?If the flow doesn't have >> enough data to trigger fast retransmit then that is unfortunate for that >> user since they now have to wait an RTT for that packet, and it does not >> drain the queue very much. >> >> >> Hence, we have the good old loss differentiation problem. And because >> CoDel is particularly intended for edge routers, the disaster is placed >> exactly there where it is expected to happen......8-) >> >> Detlef >> >> Am 01.12.2013 22:16, schrieb Andrew Mcgregor: >> > I mean sojourn time, one way in the particular queue, as per CoDel, >> rather >> > than anything TCP-related. ?Clearance rate is fairly simply related to >> > sojourn time, of course, given enough integration time for the statistics >> > to converge. >> > >> > >> > On 2 December 2013 02:57, Detlef Bosau wrote: >> > >> >> Am 01.12.2013 06:05, schrieb Andrew Mcgregor: >> >>> The actual clearance rate from the queue (or the sojourn time), if that >> >>> matters for your AQM scheme. ?That way you are not assuming a known >> line >> >>> rate. >> >> Clearance rate or sojourn time? >> >> >> >> Clearance rate may apply for a packet delivery rate. From a TCP point of >> >> view, the sojourn time is the difference between the arrival of the >> >> according ACK and the time a data packet left the sender. >> >> >> >> So you omit any recovery latency. >> >> >> >> >> >> >> >>> On 30 November 2013 00:13, Detlef Bosau wrote: >> >>> >> >>>> ?Am 29.11.2013 00:24, schrieb Andrew Mcgregor: >> >>>> >> >>>> In which case... measure, don't assume. ?Served us well for 802.11 >> >>>> modulation selection, I don't see why it shouldn't work for AQM. >> >>>> >> >>>> >> >>>> What do you want to measure? >> >>>> >> >>> >> >> >> >> -- >> >> ------------------------------------------------------------------ >> >> Detlef Bosau >> >> Galileistra?e 30 >> >> 70565 Stuttgart ? ? ? ? ? ? ? ? ? ? ? ? ? ?Tel.: ? +49 711 5208031 >> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mobile: +49 172 6819937 >> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?skype: ? ? detlef.bosau >> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ICQ: ? ? ? ? ?566129673 >> >> detlef.bosau at web.de ? ? ? ? ? ? ? ? ? ?http://www.detlef-bosau.de >> >> >> >> >> >> > >> >> >> -- >> ------------------------------------------------------------------ >> Detlef Bosau >> Galileistra?e 30 >> 70565 Stuttgart ? ? ? ? ? ? ? ? ? ? ? ? ? ?Tel.: ? +49 711 5208031 >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mobile: +49 172 6819937 >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?skype: ? ? detlef.bosau >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ICQ: ? ? ? ? ?566129673 >> detlef.bosau at web.de ? ? ? ? ? ? ? ? ? ?http://www.detlef-bosau.de >> >> >> >> >> -- >> Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 >> >> >> > > >-- >Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 > From sk.anirudh at gmail.com Tue Dec 3 18:02:14 2013 From: sk.anirudh at gmail.com (Anirudh Sivaraman) Date: Tue, 3 Dec 2013 21:02:14 -0500 Subject: [e2e] Codel and Wireless In-Reply-To: <1386120880.45379.YahooMailNeo@web141604.mail.bf1.yahoo.com> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <1386120880.45379.YahooMailNeo@web141604.mail.bf1.yahoo.com> Message-ID: On Tue, Dec 3, 2013 at 8:34 PM, Daniel Havey wrote: > On Tuesday, December 3, 2013 4:54 PM, Keith Winstein wrote: > Folks here might be interested -- my colleague Anirudh Sivaraman has done > some quantitative testing on sfqCoDel (the ns-2 module) vs. CoDel (one > queue) vs. large DropTail queues (with per-flow queueing), in ns-2 emulation > of a Verizon LTE link. > > Paper, code and replication instructions here: > http://web.mit.edu/anirudh/www/sdn-data-plane.html > > We emulate a commercial LTE link queue, where datagrams arrive at the back > whenever somebody chooses to send one, and are serviced (and delivered) at > particular instants if the queue is nonempty. We recorded those instants > from a real LTE network, by keeping the queue nonempty and then measuring > when datagrams were delivered to the other side, and then play back the same > services in emulation. (The same strategy was used in our Sprout work.) > > The summary is that we find fq_codel is pretty reasonable overall, but there > are cases where it suffers considerably on some metrics that may be worth > knowing about. For instance, the replication dataset includes a case where a > bulk flow of TCP NewReno gets 12 Mbps over LTE (with a bufferbloated link), > and only 4.4 Mbps if the LTE link is controlled with fq_codel or codel. > > There's also a case where we look at a contending bulk-throughput > application and a Web-like application going over the same link, and *both* > workloads end up doing a lot better over the bufferbloated, per-flow-queue > LTE link than the fq_codel link. > > This is with a single real-world trace that was recorded from Verizon's LTE > network in Boston. I don't think our dataset is sufficient to start making > claims about whether fq_codel is or isn't good in the real world and whether > Verizon should implement it on their base station queues (and whether > Qualcomm should implement it on their baseband LTE modem queues for the > uplink), but it could be useful as a tool to measure the kinds of properties > Andrew is talking about without having to implement CoDel on an eNodeB. > > AFAIK we haven't explored sweeping the CoDel parameters (the 100ms and the > 5ms) to see if different values perform better empirically on these traces, > but maybe we should... > > Cheers, > Keith > >>>> Nice study! I was just wondering what would happen if the CoDel >>>> parameters were not true. Specifically something like this: What happens to >>>> a connection that exceeds the 100ms RTT?| CoDel tracks queuing delay on a particular link, not the round trip time of a connection, which can depend on propagation and transmission delays on other links as well. Our experiments were run with a 150 ms minimum RTT, and CoDel is designed for wide area networks where the minimum RTT routinely exceeds 100 ms. > ...Daniel > > > On Tue, Dec 3, 2013 at 6:45 PM, Andrew Mcgregor > wrote: > > Empirically, for fq_codel, long RTT flows work fine so long as RTT < 5 > intervals, roughly speaking, and it degrades very slowly. So 100ms is > about right for the internet. > > > On 4 December 2013 10:10, Daniel Havey wrote: > >> On Tuesday, December 3, 2013 2:41 PM, Andrew Mcgregor < >> andrewmcgr at google.com> wrote: >> All of which is why fq_codel is so much better... because flows queue >> independently, and drops are calculated per flow (although overall queue >> size is included implicitly via the sojourn time), the RTT delay has far >> less impact. CoDel is an ingredient of an AQM system, not a desirable AQM >> on its own. >> >> >Makes sense to me. We need to get the worst case RTT right. If we set >> the interval to 100ms then the user with the users with larger RTTs may >> have issues. >> >> >> On 4 December 2013 08:11, Daniel Havey wrote: >> >> >> >> >> >> >> On Tuesday, December 3, 2013 12:22 PM, Detlef Bosau >> wrote: >> >> To my understanding, the "sojourn time" considered in CoDel is the >> difference between the time when a packet/leaves /a queue and the time, >> when this packet has /arrived /at the queue. In other words: The time a >> packet spends in the queue. >> >> When this time is unusually high, CoDel sees an imminent congestion and >> drops packets. >> >> The problem is that CoDel makes no difference whether the "sojourn time" >> is caused by a huge number of packets in the queue, i.e. congestion, or >> by a huge delivery time resulting from corruption loss and necessary >> retransmissions. >> >> CoDel parameters are interval and target. If the queue drains before the >> interval then there shouldn't be any drops. Also there is "leverage" >> from >> Red in a Different Light. If CoDel decides to drop a packet from a flow >> that is in congestion avoidance, fast retransmit or slow start then the >> window is halved and the queue drains quickly. If the flow doesn't have >> enough data to trigger fast retransmit then that is unfortunate for that >> user since they now have to wait an RTT for that packet, and it does not >> drain the queue very much. >> >> >> Hence, we have the good old loss differentiation problem. And because >> CoDel is particularly intended for edge routers, the disaster is placed >> exactly there where it is expected to happen......8-) >> >> Detlef >> >> Am 01.12.2013 22:16, schrieb Andrew Mcgregor: >> > I mean sojourn time, one way in the particular queue, as per CoDel, >> rather >> > than anything TCP-related. Clearance rate is fairly simply related to >> > sojourn time, of course, given enough integration time for the >> > statistics >> > to converge. >> > >> > >> > On 2 December 2013 02:57, Detlef Bosau wrote: >> > >> >> Am 01.12.2013 06:05, schrieb Andrew Mcgregor: >> >>> The actual clearance rate from the queue (or the sojourn time), if >> >>> that >> >>> matters for your AQM scheme. That way you are not assuming a known >> line >> >>> rate. >> >> Clearance rate or sojourn time? >> >> >> >> Clearance rate may apply for a packet delivery rate. From a TCP point >> >> of >> >> view, the sojourn time is the difference between the arrival of the >> >> according ACK and the time a data packet left the sender. >> >> >> >> So you omit any recovery latency. >> >> >> >> >> >> >> >>> On 30 November 2013 00:13, Detlef Bosau wrote: >> >>> >> >>>> Am 29.11.2013 00:24, schrieb Andrew Mcgregor: >> >>>> >> >>>> In which case... measure, don't assume. Served us well for 802.11 >> >>>> modulation selection, I don't see why it shouldn't work for AQM. >> >>>> >> >>>> >> >>>> What do you want to measure? >> >>>> >> >>> >> >> >> >> -- >> >> ------------------------------------------------------------------ >> >> Detlef Bosau >> >> Galileistra?e 30 >> >> 70565 Stuttgart Tel.: +49 711 5208031 >> >> mobile: +49 172 6819937 >> >> skype: detlef.bosau >> >> ICQ: 566129673 >> >> detlef.bosau at web.de http://www.detlef-bosau.de >> >> >> >> >> >> > >> >> >> -- >> ------------------------------------------------------------------ >> Detlef Bosau >> Galileistra?e 30 >> 70565 Stuttgart Tel.: +49 711 5208031 >> mobile: +49 172 6819937 >> skype: detlef.bosau >> ICQ: 566129673 >> detlef.bosau at web.de http://www.detlef-bosau.de >> >> >> >> >> -- >> Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 >> >> >> > > > -- > Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 > > > > From dhavey at yahoo.com Tue Dec 3 18:29:57 2013 From: dhavey at yahoo.com (Daniel Havey) Date: Tue, 3 Dec 2013 18:29:57 -0800 (PST) Subject: [e2e] Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <1386120880.45379.YahooMailNeo@web141604.mail.bf1.yahoo.com> Message-ID: <1386124197.37806.YahooMailNeo@web141601.mail.bf1.yahoo.com> On Tuesday, December 3, 2013 6:02 PM, Anirudh Sivaraman wrote: On Tue, Dec 3, 2013 at 8:34 PM, Daniel Havey wrote: > On Tuesday, December 3, 2013 4:54 PM, Keith Winstein wrote: > Folks here might be interested -- my colleague Anirudh Sivaraman has done > some quantitative testing on sfqCoDel (the ns-2 module) vs. CoDel (one > queue) vs. large DropTail queues (with per-flow queueing), in ns-2 emulation > of a Verizon LTE link. > > Paper, code and replication instructions here: > http://web.mit.edu/anirudh/www/sdn-data-plane.html > > We emulate a commercial LTE link queue, where datagrams arrive at the back > whenever somebody chooses to send one, and are serviced (and delivered) at > particular instants if the queue is nonempty. We recorded those instants > from a real LTE network, by keeping the queue nonempty and then measuring > when datagrams were delivered to the other side, and then play back the same > services in emulation. (The same strategy was used in our Sprout work.) > > The summary is that we find fq_codel is pretty reasonable overall, but there > are cases where it suffers considerably on some metrics that may be worth > knowing about. For instance, the replication dataset includes a case where a > bulk flow of TCP NewReno gets 12 Mbps over LTE (with a bufferbloated link), > and only 4.4 Mbps if the LTE link is controlled with fq_codel or codel. > > There's also a case where we look at a contending bulk-throughput > application and a Web-like application going over the same link, and *both* > workloads end up doing a lot better over the bufferbloated, per-flow-queue > LTE link than the fq_codel link. > > This is with a single real-world trace that was recorded from Verizon's LTE > network in Boston. I don't think our dataset is sufficient to start making > claims about whether fq_codel is or isn't good in the real world and whether > Verizon should implement it on their base station queues (and whether > Qualcomm should implement it on their baseband LTE modem queues for the > uplink), but it could be useful as a tool to measure the kinds of properties > Andrew is talking about without having to implement CoDel on an eNodeB. > > AFAIK we haven't explored sweeping the CoDel parameters (the 100ms and the > 5ms) to see if different values perform better empirically on these traces, > but maybe we should... > > Cheers, > Keith > >>>> Nice study!? I was just wondering what would happen if the CoDel >>>> parameters were not true.? Specifically something like this: What happens to >>>> a connection that exceeds the 100ms RTT?| CoDel tracks queuing delay on a particular link, not the round trip time of a connection, which can depend on propagation and transmission delays on other links as well. Our experiments were run with a 150 ms minimum RTT, and CoDel is designed for wide area networks where the minimum RTT routinely exceeds 100 ms. >> This makes a lot of sense. ?When I read V&K's Controlling Queue Delay paper they seemed to imply that CoDel works really well for RTTs from 10-300ms, with acceptable performance up to 1 second. > ...Daniel > > > On Tue, Dec 3, 2013 at 6:45 PM, Andrew Mcgregor > wrote: > > Empirically, for fq_codel, long RTT flows work fine so long as RTT < 5 > intervals, roughly speaking, and it degrades very slowly.? So 100ms is > about right for the internet. > > > On 4 December 2013 10:10, Daniel Havey wrote: > >> On Tuesday, December 3, 2013 2:41 PM, Andrew Mcgregor < >> andrewmcgr at google.com> wrote: >>? All of which is why fq_codel is so much better... because flows queue >> independently, and drops are calculated per flow (although overall queue >> size is included implicitly via the sojourn time), the RTT delay has far >> less impact.? CoDel is an ingredient of an AQM system, not a desirable AQM >> on its own. >> >> >Makes sense to me.? We need to get the worst case RTT right.? If we set >> the interval to 100ms then the user with the users with larger RTTs may >> have issues. >> >> >> On 4 December 2013 08:11, Daniel Havey wrote: >> >> >> >> >> >> >> On Tuesday, December 3, 2013 12:22 PM, Detlef Bosau >> wrote: >> >> To my understanding, the "sojourn time" considered in CoDel is the >> difference between the time when a packet/leaves /a queue and the time, >> when this packet has /arrived /at the queue. In other words: The time a >> packet spends in the queue. >> >> When this time is unusually high, CoDel sees an imminent congestion and >> drops packets. >> >> The problem is that CoDel makes no difference whether the "sojourn time" >> is caused by a huge number of packets in the queue, i.e. congestion, or >> by a huge delivery time resulting from corruption loss and necessary >> retransmissions. >> >> CoDel parameters are interval and target.? If the queue drains before the >> interval then there shouldn't be any drops.? Also there is "leverage" >> from >> Red in a Different Light.? If CoDel decides to drop a packet from a flow >> that is in congestion avoidance, fast retransmit or slow start then the >> window is halved and the queue drains quickly.? If the flow doesn't have >> enough data to trigger fast retransmit then that is unfortunate for that >> user since they now have to wait an RTT for that packet, and it does not >> drain the queue very much. >> >> >> Hence, we have the good old loss differentiation problem. And because >> CoDel is particularly intended for edge routers, the disaster is placed >> exactly there where it is expected to happen......8-) >> >> Detlef >> >> Am 01.12.2013 22:16, schrieb Andrew Mcgregor: >> > I mean sojourn time, one way in the particular queue, as per CoDel, >> rather >> > than anything TCP-related.? Clearance rate is fairly simply related to >> > sojourn time, of course, given enough integration time for the >> > statistics >> > to converge. >> > >> > >> > On 2 December 2013 02:57, Detlef Bosau wrote: >> > >> >> Am 01.12.2013 06:05, schrieb Andrew Mcgregor: >> >>> The actual clearance rate from the queue (or the sojourn time), if >> >>> that >> >>> matters for your AQM scheme.? That way you are not assuming a known >> line >> >>> rate. >> >> Clearance rate or sojourn time? >> >> >> >> Clearance rate may apply for a packet delivery rate. From a TCP point >> >> of >> >> view, the sojourn time is the difference between the arrival of the >> >> according ACK and the time a data packet left the sender. >> >> >> >> So you omit any recovery latency. >> >> >> >> >> >> >> >>> On 30 November 2013 00:13, Detlef Bosau wrote: >> >>> >> >>>>? Am 29.11.2013 00:24, schrieb Andrew Mcgregor: >> >>>> >> >>>> In which case... measure, don't assume.? Served us well for 802.11 >> >>>> modulation selection, I don't see why it shouldn't work for AQM. >> >>>> >> >>>> >> >>>> What do you want to measure? >> >>>> >> >>> >> >> >> >> -- >> >> ------------------------------------------------------------------ >> >> Detlef Bosau >> >> Galileistra?e 30 >> >> 70565 Stuttgart? ? ? ? ? ? ? ? ? ? ? ? ? ? Tel.:? +49 711 5208031 >> >>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? mobile: +49 172 6819937 >> >>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? skype:? ? detlef.bosau >> >>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ICQ:? ? ? ? ? 566129673 >> >> detlef.bosau at web.de? ? ? ? ? ? ? ? ? ? http://www.detlef-bosau.de >> >> >> >> >> >> > >> >> >> -- >> ------------------------------------------------------------------ >> Detlef Bosau >> Galileistra?e 30 >> 70565 Stuttgart? ? ? ? ? ? ? ? ? ? ? ? ? ? Tel.:? +49 711 5208031 >>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? mobile: +49 172 6819937 >>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? skype:? ? detlef.bosau >>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ICQ:? ? ? ? ? 566129673 >> detlef.bosau at web.de? ? ? ? ? ? ? ? ? ? http://www.detlef-bosau.de >> >> >> >> >> -- >> Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 >> >> >> > > > -- > Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 > > > > From Emmanuel.LOCHIN at isae.fr Tue Dec 3 23:10:51 2013 From: Emmanuel.LOCHIN at isae.fr (LOCHIN Emmanuel) Date: Wed, 04 Dec 2013 08:10:51 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: Message-ID: <47c4-529ed580-12d-4f229700@253745013> Le Mercredi 4 D?cembre 2013 00:45 CET, Andrew Mcgregor a ?crit: > Empirically, for fq_codel, long RTT flows work fine so long as RTT < 5 > intervals, roughly speaking, and it degrades very slowly. So 100ms is > about right for the internet. Dear all, I have questions about this value that appears everywhere but doesn't seem to have any scientific support (I mean I didn't found something about it). It would be great if somebody could give me any evidence (serious statistical study, scientific paper, ...) about this value explaining where does it come from. How this value has been chosen and what about people running over long delay such as satellite broadband access? Emmanuel > On 4 December 2013 10:10, Daniel Havey wrote: > > > On Tuesday, December 3, 2013 2:41 PM, Andrew Mcgregor < > > andrewmcgr at google.com> wrote: > > All of which is why fq_codel is so much better... because flows queue > > independently, and drops are calculated per flow (although overall queue > > size is included implicitly via the sojourn time), the RTT delay has far > > less impact. CoDel is an ingredient of an AQM system, not a desirable AQM > > on its own. > > > > >Makes sense to me. We need to get the worst case RTT right. If we set > > the interval to 100ms then the user with the users with larger RTTs may > > have issues. > > > > > > On 4 December 2013 08:11, Daniel Havey wrote: > > > > > > > > > > > > > > On Tuesday, December 3, 2013 12:22 PM, Detlef Bosau > > wrote: > > > > To my understanding, the "sojourn time" considered in CoDel is the > > difference between the time when a packet/leaves /a queue and the time, > > when this packet has /arrived /at the queue. In other words: The time a > > packet spends in the queue. > > > > When this time is unusually high, CoDel sees an imminent congestion and > > drops packets. > > > > The problem is that CoDel makes no difference whether the "sojourn time" > > is caused by a huge number of packets in the queue, i.e. congestion, or > > by a huge delivery time resulting from corruption loss and necessary > > retransmissions. > > > > CoDel parameters are interval and target. If the queue drains before the > > interval then there shouldn't be any drops. Also there is "leverage" from > > Red in a Different Light. If CoDel decides to drop a packet from a flow > > that is in congestion avoidance, fast retransmit or slow start then the > > window is halved and the queue drains quickly. If the flow doesn't have > > enough data to trigger fast retransmit then that is unfortunate for that > > user since they now have to wait an RTT for that packet, and it does not > > drain the queue very much. > > > > > > Hence, we have the good old loss differentiation problem. And because > > CoDel is particularly intended for edge routers, the disaster is placed > > exactly there where it is expected to happen......8-) > > > > Detlef > > > > Am 01.12.2013 22:16, schrieb Andrew Mcgregor: > > > I mean sojourn time, one way in the particular queue, as per CoDel, > > rather > > > than anything TCP-related. Clearance rate is fairly simply related to > > > sojourn time, of course, given enough integration time for the statistics > > > to converge. > > > > > > > > > On 2 December 2013 02:57, Detlef Bosau wrote: > > > > > >> Am 01.12.2013 06:05, schrieb Andrew Mcgregor: > > >>> The actual clearance rate from the queue (or the sojourn time), if that > > >>> matters for your AQM scheme. That way you are not assuming a known > > line > > >>> rate. > > >> Clearance rate or sojourn time? > > >> > > >> Clearance rate may apply for a packet delivery rate. From a TCP point of > > >> view, the sojourn time is the difference between the arrival of the > > >> according ACK and the time a data packet left the sender. > > >> > > >> So you omit any recovery latency. > > >> > > >> > > >> > > >>> On 30 November 2013 00:13, Detlef Bosau wrote: > > >>> > > >>>> Am 29.11.2013 00:24, schrieb Andrew Mcgregor: > > >>>> > > >>>> In which case... measure, don't assume. Served us well for 802.11 > > >>>> modulation selection, I don't see why it shouldn't work for AQM. > > >>>> > > >>>> > > >>>> What do you want to measure? > > >>>> > > >>> > > >> > > >> -- > > >> ------------------------------------------------------------------ > > >> Detlef Bosau > > >> Galileistra?e 30 > > >> 70565 Stuttgart Tel.: +49 711 5208031 > > >> mobile: +49 172 6819937 > > >> skype: detlef.bosau > > >> ICQ: 566129673 > > >> detlef.bosau at web.de http://www.detlef-bosau.de > > > > >> > > >> > > > > > > > > > -- > > ------------------------------------------------------------------ > > Detlef Bosau > > Galileistra?e 30 > > 70565 Stuttgart Tel.: +49 711 5208031 > > mobile: +49 172 6819937 > > skype: detlef.bosau > > ICQ: 566129673 > > detlef.bosau at web.de http://www.detlef-bosau.de > > > > > > > > > > -- > > Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 > > > > > > > > > -- > Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 From detlef.bosau at web.de Wed Dec 4 04:11:35 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Wed, 04 Dec 2013 13:11:35 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> Message-ID: <529F1BF7.4070702@web.de> Am 03.12.2013 22:11, schrieb Daniel Havey: > CoDel parameters are interval and target. Particularly "interval" is a yet to be discovered fundamental physical constant, the Nobel price and the Turing award for which are still pending. (Less pathetically spoken: It is a constant discovered by three or four experiments and now, it is "the" fundamental constant which is valid in the whole universe. When I did so in papers, these papers were rejected, without exception. What are we pursuing? Science or gambling?) > If the queue drains before the interval then there shouldn't be any drops. Yes, Daniel. I know the code. And when I claim this is nonsense, then I'm first ready to take flames and send, you can be assured that I read the code and the comments and papers VERY MUCH more than once. And even in this very moment, I have the pseudocode visible on the other screen. And actually, the "sojourn time" is a qeue sojourn time and it is calculated from time stamps, where the "clock" is read when the packet is enqueued and dequeued. With respect to queueing systems: CoDel measures queueing time, CoDel doesn't meaure sojourn time. (:= Queueing Time + Service Time as you remember from lectures.) -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From detlef.bosau at web.de Wed Dec 4 09:40:23 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Wed, 04 Dec 2013 18:40:23 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> Message-ID: <529F6907.7070003@web.de> Am 03.12.2013 23:41, schrieb Andrew Mcgregor: > All of which is why fq_codel is so much better. than what? Either you mixed up queueing time and sojourn time or you don't really understand what I would like to talk about. Frankly spoken, I'm a bit annoyed about this discussion. -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From dhavey at yahoo.com Wed Dec 4 12:21:23 2013 From: dhavey at yahoo.com (Daniel Havey) Date: Wed, 4 Dec 2013 12:21:23 -0800 (PST) Subject: [e2e] Codel and Wireless In-Reply-To: <529F1BF7.4070702@web.de> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F1BF7.4070702@web.de> Message-ID: <1386188483.43698.YahooMailNeo@web141601.mail.bf1.yahoo.com> On Wednesday, December 4, 2013 4:42 AM, Detlef Bosau wrote: Am 03.12.2013 22:11, schrieb Daniel Havey: > CoDel parameters are interval and target.? Particularly "interval" is a yet to be discovered fundamental physical constant, the Nobel price and the Turing award for which are still pending. >> Haha! ?And the answer to life, the universe and everything is 42! ?Anyways I read a really nice paper about a really cool parameterless algorithm (CoDel) and I find that it has 2 parameters. ?Huh?! ?Then I read on and find out that they are not parameters, they are magic numbers! ?Now I really want to know! ?I am ready to believe that 100ms and 5ms are pretty darn good numbers. ?But the paper didn't explain why. ?I still want to know why. ?Clearly the interval value affects response time. ?What else? ? And finally the pragmatic question. ?Just like Emmanuel asked. ?Do these numbers work over a slow laggy satellite link. ?I think the parameters work well over a wide range of scenarios just like it says in the paper, but, if you put that thing on my colleague's connection it will kill you. The connection is a 128 Kbps with burst rates up to 256 Kbps satellite and we have measured 10 seconds of ping time under load. ?Clearly 100ms and 5ms will perform miserably on this link since >5ms standing queue time is normal performance. ?The interval value is suspect here too. ?One hundred milliseconds? ?Give me a break! ?We have 500 ms just in flight time to outer space and back. Now that I am done with my rant, let me just say that I like CoDel and I think fair queuing is cool. ?Heck yes I want a queue for every flow! ?Who would say no to that? ?But I will (and did) recommend to my African friend not to use those numbers. ?That is my disclaimer since I am not as flameproof as you! ?;^) ...Daniel (Less pathetically spoken: It is a constant discovered by three or four experiments and now, it is "the" fundamental constant which is valid in the whole universe. When I did so in papers, these papers were rejected, without exception. What are we pursuing? Science or gambling?) > If the queue drains before the interval then there shouldn't be any drops. Yes, Daniel. I know the code. And when I claim this is nonsense, then I'm first ready to take flames and send, you can be assured that I read the code and the comments and papers VERY MUCH more than once. And even in this very moment, I have the pseudocode visible on the other screen. And actually, the "sojourn time" is a qeue sojourn time and it is calculated from time stamps, where the "clock" is read when the packet is enqueued and dequeued. With respect to queueing systems: CoDel measures queueing time, CoDel doesn't meaure sojourn time. (:= Queueing Time + Service Time as you remember from lectures.) -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30? 70565 Stuttgart? ? ? ? ? ? ? ? ? ? ? ? ? ? Tel.:???+49 711 5208031 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ???mobile: +49 172 6819937 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ???skype:? ???detlef.bosau ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ???ICQ:? ? ? ? ? 566129673 detlef.bosau at web.dehttp://www.detlef-bosau.de? ? ? ? ? ? ? ? ? ? From detlef.bosau at web.de Wed Dec 4 12:26:52 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Wed, 04 Dec 2013 21:26:52 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: <1386183322.935517047@apps.rackspace.com> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <1386176840.976127048@apps.rackspace.com> <1386183322.935517047@apps.rackspace.com> Message-ID: <529F900C.7080306@web.de> Am 04.12.2013 19:55, schrieb dpreed at reed.com: > > I'd like to understand how web traffic does better on a fully > congested link. > > > better than what? > > Is it possible that you define "bufferbloated" to mean that there is > at least one queue that is very large? > To my understanding, buffer bloat is exactly that. > > > > If in that case a short "web traffic" queue gets priority, and there > is minimal physical queueing in the devices that can get filled - that > would not be "buffer bloated" according to any definition I know of. > When there are no devices that can be filled there is hardly any buffer bloat. > > > However, identifying "web traffic" (or "game click" traffic) at the > switch/router level is a serious violation of modularity (sometimes > called layer violation). We used to call that a "kludge". > Shall the network offer service to the users? Or shall the network offer service to the layers? As you know, Ford & Iyengar put the traditional layer concept in question. That does not mean that layers as they are defined are bad. But that does mean, that layers aren't sacred cows. Sacred cows are a matter of religion. Not a matter of science. > > > And from a technical point of view, on an end-to-end basis, having a > seriously backed up queue of one class of traffic in the middle of the > network is never a good idea - it provides no extra throughput, and > while prioritized traffic is little disturbed, the users of that queue > are essentially making the network unable to reallocate physical layer > resources (e.g. the modulated, shared channel). > Fine. So it might make sense to handle different traffic in different ways? > > > Why should a queue be longer than 2 packets at any point in the > network, at any point in time? > I argued several times why in wireless networks some amount of queueing may make sense. And I argued, that this is a non trivial issue which requires some consideration. I would be a happier man than I am, when all questions in life could be sufficiently answered by one simple answer. And the question you ask is one of the apparently simple questions which may require a not so simple answer. -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From andrewmcgr at google.com Wed Dec 4 14:52:23 2013 From: andrewmcgr at google.com (Andrew Mcgregor) Date: Thu, 5 Dec 2013 09:52:23 +1100 Subject: [e2e] Codel and Wireless In-Reply-To: <529F6907.7070003@web.de> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> Message-ID: CoDel has a definition of 'sojourn time', which is the way Van and Kathie defined it. You're using a different definition, and therefore we are talking at cross purposes. The CoDel definition is equivalent to 'queue residence time'. I don't know what you mean by 'sojourn time', but whatever, it's irrelevant to discussion of CoDel. On 5 December 2013 04:40, Detlef Bosau wrote: > Am 03.12.2013 23:41, schrieb Andrew Mcgregor: > > All of which is why fq_codel is so much better. > > than what? > > Either you mixed up queueing time and sojourn time or you don't really > understand what I would like to talk about. > > Frankly spoken, I'm a bit annoyed about this discussion. > > > -- > ------------------------------------------------------------------ > Detlef Bosau > Galileistra?e 30 > 70565 Stuttgart Tel.: +49 711 5208031 > mobile: +49 172 6819937 > skype: detlef.bosau > ICQ: 566129673 > detlef.bosau at web.de http://www.detlef-bosau.de > > -- Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 From Nicolas.Kuhn at isae.fr Thu Dec 5 00:44:11 2013 From: Nicolas.Kuhn at isae.fr (KUHN Nicolas) Date: Thu, 05 Dec 2013 09:44:11 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: <47c4-529ed580-12d-4f229700@253745013> References: <47c4-529ed580-12d-4f229700@253745013> Message-ID: Dear all, These discussions remind me a lot about the target value of LEDBAT. Indeed, without clear comparative studies [at least from what I know - I would be glad to be wrong], the target was said in the RFC to be set up to 100 ms. However, we measured that in a long delay path, these 100ms were not optimal [0]. Although the setting the target value to 5 ms is not optimal neither, we believe that one should not claim the choice for a value for a parameter without clear evaluations. As a result, we would like to see evaluations of the impact of these interval and target on the performance of CoDel. Indeed, it makes no sense to me to set them without justifications. As an example, in a document published by CableLabs [1], the authors explain that " Since we measure sojourn time such that it includes this MAC latency, we simply increased the value for target (to 10 ms in our experiments) such that a packet arriving to an empty queue would be assured (absent RF congestion) that it would have a sojourn time less than the target value.". Even though this issue may be solved by sf_codel, there is a clear lack of evaluations of fq_codel as well: do sfq and codel mix well together ? A clear comparison between CoDel and ARED should not only discuss the gain in 'queuing-delay', but also discuss the deployment issues that CoDel may encounter if it is deployed at a large scale with arbitrary set parameters. Nicolas [0] http://www.nicta.com.au/pub?id=6691 [1] ACTIVE QUEUE MANAGEMENT ALGORITHMS FOR DOCSIS 3.0 - A Simulation Study of CoDel, SFQ-CoDel and PIE in DOCSIS 3.0 Networks. 2013 Le 04.12.2013 08:10, LOCHIN Emmanuel a ?crit?: > Le Mercredi 4 D?cembre 2013 00:45 CET, Andrew Mcgregor > a ?crit: > >> Empirically, for fq_codel, long RTT flows work fine so long as RTT < >> 5 >> intervals, roughly speaking, and it degrades very slowly. So 100ms >> is >> about right for the internet. > > Dear all, > > I have questions about this value that appears everywhere but doesn't > seem to have any scientific support (I mean I didn't found something > about it). It would be great if somebody could give me any evidence > (serious statistical study, scientific paper, ...) about this value > explaining where does it come from. How this value has been chosen > and > what about people running over long delay such as satellite broadband > access? > > Emmanuel > > >> On 4 December 2013 10:10, Daniel Havey wrote: >> >> > On Tuesday, December 3, 2013 2:41 PM, Andrew Mcgregor < >> > andrewmcgr at google.com> wrote: >> > All of which is why fq_codel is so much better... because flows >> queue >> > independently, and drops are calculated per flow (although overall >> queue >> > size is included implicitly via the sojourn time), the RTT delay >> has far >> > less impact. CoDel is an ingredient of an AQM system, not a >> desirable AQM >> > on its own. >> > >> > >Makes sense to me. We need to get the worst case RTT right. If >> we set >> > the interval to 100ms then the user with the users with larger >> RTTs may >> > have issues. >> > >> > >> > On 4 December 2013 08:11, Daniel Havey wrote: >> > >> > >> > >> > >> > >> > >> > On Tuesday, December 3, 2013 12:22 PM, Detlef Bosau >> >> > wrote: >> > >> > To my understanding, the "sojourn time" considered in CoDel is the >> > difference between the time when a packet/leaves /a queue and the >> time, >> > when this packet has /arrived /at the queue. In other words: The >> time a >> > packet spends in the queue. >> > >> > When this time is unusually high, CoDel sees an imminent >> congestion and >> > drops packets. >> > >> > The problem is that CoDel makes no difference whether the "sojourn >> time" >> > is caused by a huge number of packets in the queue, i.e. >> congestion, or >> > by a huge delivery time resulting from corruption loss and >> necessary >> > retransmissions. >> > >> > CoDel parameters are interval and target. If the queue drains >> before the >> > interval then there shouldn't be any drops. Also there is >> "leverage" from >> > Red in a Different Light. If CoDel decides to drop a packet from >> a flow >> > that is in congestion avoidance, fast retransmit or slow start >> then the >> > window is halved and the queue drains quickly. If the flow >> doesn't have >> > enough data to trigger fast retransmit then that is unfortunate >> for that >> > user since they now have to wait an RTT for that packet, and it >> does not >> > drain the queue very much. >> > >> > >> > Hence, we have the good old loss differentiation problem. And >> because >> > CoDel is particularly intended for edge routers, the disaster is >> placed >> > exactly there where it is expected to happen......8-) >> > >> > Detlef >> > >> > Am 01.12.2013 22:16, schrieb Andrew Mcgregor: >> > > I mean sojourn time, one way in the particular queue, as per >> CoDel, >> > rather >> > > than anything TCP-related. Clearance rate is fairly simply >> related to >> > > sojourn time, of course, given enough integration time for the >> statistics >> > > to converge. >> > > >> > > >> > > On 2 December 2013 02:57, Detlef Bosau >> wrote: >> > > >> > >> Am 01.12.2013 06:05, schrieb Andrew Mcgregor: >> > >>> The actual clearance rate from the queue (or the sojourn >> time), if that >> > >>> matters for your AQM scheme. That way you are not assuming a >> known >> > line >> > >>> rate. >> > >> Clearance rate or sojourn time? >> > >> >> > >> Clearance rate may apply for a packet delivery rate. From a TCP >> point of >> > >> view, the sojourn time is the difference between the arrival of >> the >> > >> according ACK and the time a data packet left the sender. >> > >> >> > >> So you omit any recovery latency. >> > >> >> > >> >> > >> >> > >>> On 30 November 2013 00:13, Detlef Bosau >> wrote: >> > >>> >> > >>>> Am 29.11.2013 00:24, schrieb Andrew Mcgregor: >> > >>>> >> > >>>> In which case... measure, don't assume. Served us well for >> 802.11 >> > >>>> modulation selection, I don't see why it shouldn't work for >> AQM. >> > >>>> >> > >>>> >> > >>>> What do you want to measure? >> > >>>> >> > >>> >> > >> >> > >> -- >> > >> >> ------------------------------------------------------------------ >> > >> Detlef Bosau >> > >> Galileistra?e 30 >> > >> 70565 Stuttgart Tel.: +49 711 >> 5208031 >> > >> mobile: +49 172 >> 6819937 >> > >> skype: >> detlef.bosau >> > >> ICQ: >> 566129673 >> > >> detlef.bosau at web.de >> http://www.detlef-bosau.de >> > >> > >> >> > >> >> > > >> > >> > >> > -- >> > ------------------------------------------------------------------ >> > Detlef Bosau >> > Galileistra?e 30 >> > 70565 Stuttgart Tel.: +49 711 5208031 >> > mobile: +49 172 6819937 >> > skype: detlef.bosau >> > ICQ: 566129673 >> > detlef.bosau at web.de http://www.detlef-bosau.de >> > >> > >> > >> > >> > -- >> > Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 >> > >> > >> > >> >> >> -- >> Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 From detlef.bosau at web.de Thu Dec 5 04:11:55 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Thu, 05 Dec 2013 13:11:55 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> Message-ID: <52A06D8B.1030500@web.de> Daniel, as you know, GPRS allows 0.95 quantiles for 1024 byte packets up to about 10 minutes. And that's the annoying point in this discussion. Perhaps, we should have a look at the standadards. At least occasionally. Now: Can't we use GPRS then? No, or better: Obama. Yes We Can! Yes We Can! Why? Of course GPRS employs timeouts. However GPRS employs a radio block protocol which logs the round trip times for radio blocks in a GPRS link. And as you know further, GSM requires a "Timing Advance" to have all mobiles send their data timely according to the assigned timeslot and their is a certain timing advance for the uplink with respect to the downlink. (For NSA readers: Feel free to use this information for mobile location in GSM networks, the German police does so for about 20 years now. You are welcome to ask for information.) Hence, at link layer GPRS can successfully work with timeouts - perhaps not very efficiently because these timeouts perhaps are not even dynamic, I don't know. Nevertheless, as soon als you have at least one mobile link in the path, the end to end delivery time is NOT a stationary random process and therefore the use of ARMA or EWMA filters or what ever does NOT render an estimate for the process' limit pdf's expectation. Or, put in very simple and very drastic words (and dressing myself in my fire protection suite): The use of EWMA or RTT filters to obtain STT or RTT values for L 3 packets in paths containing even a single mobile link is NONSENSE. (These filters will well provide a number. However, the numbers are of no use, it's the classical "GIGO" system. Garbage In Garbage Out.) Admittedly, my intention is absolutely not to focus too much on mobile networks, because this discussion is as boaring as useless. For 14 years now, I see two parties: The CS guys who ignore reality and the reality and CE and EE guys who ignore the CS guys. I know that this sounds arrogant. However, after dealing with this matter for 14 years know and heaving read hundreds of papers concerning this matter, I simply dare the statement: Both, Lixia Zhang and Raj Jain were correct, when they stated (independently) that timers (and they discussed e2e timers as used in TCP) simply don't work, so I don't make a new claim but simply confirm what is known to the community for about 25 years. And particularly for mobile networks, I talk to colleagues and after the third mug of coffee the say: I know that it doesn't work, nevertheless I do it that way. And perhaps there is some additional fingerpointing to bad implementations of GPRS or some lousy excuses, however, and please allow me this word at least for this one moment: Bullshit is Bullshit and Bullshit remains Bullshit. Particularly the hope for a "worst case RTT" in wireless networks is simply ridiculous. And of no interest at all. Although I personally use to think networks "bottom up", networks are supposed to be designed top down: The network serves the user, it must not be the other way round. So it is simply not the question what a worst case RTT in a network could be - the only question is: What is the user's requirement? And sometimes, it requires a brave and tough and strong and wise guy to tell a user: I'm sorry, but I cannot fulfill your requirements. Or as you know from the famous RFC: no matter how you push or pull, no matter which priority your project may have - you cannot increase the speed of light! So, when there is at least one "killer argument" (terrible Germish, but you certainly understand what I mean) against a strict end 2 end attitude in TCP, this argument is called "timeouts" or "round trip times" respectively. Am 04.12.2013 00:10, schrieb Daniel Havey: > On Tuesday, December 3, 2013 2:41 PM, Andrew Mcgregor > wrote: > All of which is why fq_codel is so much better... because flows queue > independently, and drops are calculated per flow (although overall > queue size is included implicitly via the sojourn time), the RTT delay > has far less impact. CoDel is an ingredient of an AQM system, not a > desirable AQM on its own. > > >Makes sense to me. We need to get the worst case RTT right. If we > set the interval to 100ms then the user with the users with larger > RTTs may have issues. > > On 4 December 2013 08:11, Daniel Havey > wrote: > > > > > > > On Tuesday, December 3, 2013 12:22 PM, Detlef Bosau > > wrote: > > To my understanding, the "sojourn time" considered in CoDel is the > difference between the time when a packet/leaves /a queue and the > time, > when this packet has /arrived /at the queue. In other words: The > time a > packet spends in the queue. > > When this time is unusually high, CoDel sees an imminent > congestion and > drops packets. > > The problem is that CoDel makes no difference whether the "sojourn > time" > is caused by a huge number of packets in the queue, i.e. > congestion, or > by a huge delivery time resulting from corruption loss and necessary > retransmissions. > > CoDel parameters are interval and target. If the queue drains > before the interval then there shouldn't be any drops. Also > there is "leverage" from Red in a Different Light. If CoDel > decides to drop a packet from a flow that is in congestion > avoidance, fast retransmit or slow start then the window is halved > and the queue drains quickly. If the flow doesn't have enough > data to trigger fast retransmit then that is unfortunate for that > user since they now have to wait an RTT for that packet, and it > does not drain the queue very much. > > > Hence, we have the good old loss differentiation problem. And because > CoDel is particularly intended for edge routers, the disaster is > placed > exactly there where it is expected to happen......8-) > > Detlef > > Am 01.12.2013 22:16, schrieb Andrew Mcgregor: > > I mean sojourn time, one way in the particular queue, as per > CoDel, rather > > than anything TCP-related. Clearance rate is fairly simply > related to > > sojourn time, of course, given enough integration time for the > statistics > > to converge. > > > > > > On 2 December 2013 02:57, Detlef Bosau > wrote: > > > >> Am 01.12.2013 06:05, schrieb Andrew Mcgregor: > >>> The actual clearance rate from the queue (or the sojourn > time), if that > >>> matters for your AQM scheme. That way you are not assuming a > known line > >>> rate. > >> Clearance rate or sojourn time? > >> > >> Clearance rate may apply for a packet delivery rate. From a TCP > point of > >> view, the sojourn time is the difference between the arrival of the > >> according ACK and the time a data packet left the sender. > >> > >> So you omit any recovery latency. > >> > >> > >> > >>> On 30 November 2013 00:13, Detlef Bosau > wrote: > >>> > >>>> Am 29.11.2013 00:24, schrieb Andrew Mcgregor: > >>>> > >>>> In which case... measure, don't assume. Served us well for > 802.11 > >>>> modulation selection, I don't see why it shouldn't work for AQM. > >>>> > >>>> > >>>> What do you want to measure? > >>>> > >>> > >> > >> -- > >> ------------------------------------------------------------------ > >> Detlef Bosau > >> Galileistra?e 30 > >> 70565 Stuttgart Tel.: +49 711 5208031 > >> mobile: +49 172 6819937 > >> skype: detlef.bosau > >> ICQ: 566129673 > >> detlef.bosau at web.de > http://www.detlef-bosau.de > > >> > >> > > > > > -- > ------------------------------------------------------------------ > Detlef Bosau > Galileistra?e 30 > 70565 Stuttgart Tel.: +49 711 5208031 > mobile: +49 172 6819937 > skype: detlef.bosau > ICQ: 566129673 > detlef.bosau at web.de > http://www.detlef-bosau.de > > > > > -- > Andrew McGregor | SRE | andrewmcgr at google.com > | +61 4 8143 7128 > > -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From detlef.bosau at web.de Thu Dec 5 05:47:31 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Thu, 05 Dec 2013 14:47:31 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> Message-ID: <52A083F3.7000205@web.de> Keith, as long as we don't have compelling wireless link models in the ns-2, I don't think simulations in this area are useful. Particularly, quantitative results from those simulations are extremely questionable. -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From detlef.bosau at web.de Thu Dec 5 07:49:51 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Thu, 05 Dec 2013 16:49:51 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> Message-ID: <52A0A09F.40703@web.de> Am 04.12.2013 23:52, schrieb Andrew Mcgregor: > CoDel has a definition of 'sojourn time', which is the way Van and Kathie > defined it. You're using a different definition, and therefore we are > talking at cross purposes. > > The CoDel definition is equivalent to 'queue residence time'. I don't know > what you mean by 'sojourn time', Then you should have a look at an arbitrary text book on queueing theory. Basically, the queueing residence time, you're talking about, and I had a closer look at the CoDel code yeserterday so I know at least what CoDel is talking about here, is simply not feasible to assess a links load situation, particularly in mobile networks. I succumbed to this fallacy myself - and it took me years to admit my error. I've seen even PhD theses based on this nonsense, however in science we should some day make a difference between truth and fallacy. And using queueing times as a means for network load assessment is a fallacy. Perhaps, I'm completely isolated with my position - I can't help it. I have no academic affiliation and no academic contacts here in Germany, so perhaps I'm about to completely ruin my reputation here, however to my understanding, science deals with truth and error. And CoDel as a means of queue management in wireless networks is an error. -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From hari at csail.mit.edu Thu Dec 5 08:30:42 2013 From: hari at csail.mit.edu (Hari Balakrishnan) Date: Thu, 5 Dec 2013 11:30:42 -0500 Subject: [e2e] Codel and Wireless In-Reply-To: <52A083F3.7000205@web.de> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <52A083F3.7000205@web.de> Message-ID: <930C8FF4-2469-4258-A6CA-882789182A79@csail.mit.edu> Detlef, May I suggest that you please look at the paper? The simulation experiments don't use a wireless link model but are obtained over packet delivery traces collected over multiple different cellular networks (in both directions). Hari On Dec 5, 2013, at 8:47 AM, Detlef Bosau wrote: > Keith, > > as long as we don't have compelling wireless link models in the ns-2, I > don't think simulations in this area are useful. > > Particularly, quantitative results from those simulations are extremely > questionable. > > -- > ------------------------------------------------------------------ > Detlef Bosau > Galileistra?e 30 > 70565 Stuttgart Tel.: +49 711 5208031 > mobile: +49 172 6819937 > skype: detlef.bosau > ICQ: 566129673 > detlef.bosau at web.de http://www.detlef-bosau.de > From andrewmcgr at google.com Thu Dec 5 15:22:33 2013 From: andrewmcgr at google.com (Andrew Mcgregor) Date: Fri, 6 Dec 2013 10:22:33 +1100 Subject: [e2e] Codel and Wireless In-Reply-To: <52A0A09F.40703@web.de> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> Message-ID: On 6 December 2013 02:49, Detlef Bosau wrote: > Am 04.12.2013 23:52, schrieb Andrew Mcgregor: > > CoDel has a definition of 'sojourn time', which is the way Van and Kathie > > defined it. You're using a different definition, and therefore we are > > talking at cross purposes. > > > > The CoDel definition is equivalent to 'queue residence time'. I don't > know > > what you mean by 'sojourn time', > > Then you should have a look at an arbitrary text book on queueing theory. > I have, and I still don't know what you mean. But 'queue residence time' has us on the same page now, so let's just ignore sojourn as a term. > Basically, the queueing residence time, you're talking about, and I had > a closer look at the CoDel code yeserterday so I know at least what > CoDel is talking about here, is simply not feasible to assess a links > load situation, particularly in mobile networks. > Of course it isn't a measure of load, but that is not what CoDel is trying to do with it, still less fq_codel. What these algorithms are doing is effectively saying, we know that excess queueing delay (due to TCP windowed flow control) is a problem in and of itself, so we shall measure and control it directly. > I succumbed to this fallacy myself - and it took me years to admit my > error. > > I've seen even PhD theses based on this nonsense, however in science we > should some day make a difference between truth and fallacy. And using > queueing times as a means for network load assessment is a fallacy. > I agree entirely, it doesn't work for that purpose. But that's not what the AQM is aiming for. > Perhaps, I'm completely isolated with my position - I can't help it. > > I have no academic affiliation and no academic contacts here in Germany, > so perhaps I'm about to completely ruin my reputation here, however to > my understanding, science deals with truth and error. And CoDel as a > means of queue management in wireless networks is an error. I don't think so, not empirically, so long as we agree that it's not about load but about direct management of queueing delay and nothing more. However, I expect there's better ways out there, and the purpose of the AQM working group is, at least partly, to find out what they are. There's also little theoretical support for CoDel, still less fq_codel, so fq_codel has the empirical status of 'works really well in the (many) tests we have done', but not the theoretical support of 'and we know exactly why it works and what its limitations are analytically'. I hope we can find some algorithm with both, but I'm not holding my breath. By the way, fq_codel is not a combination of SFQ and CoDel, but something else. The code deserves really close study to understand what is really going on there, because there is a very interesting contribution in terms of heuristically measuring whether flows are building queue in the qdisc or not. Empirically, this works really well on user access links, including LTE, by my own direct observation having run it on my home network's LTE link for many months now. -- Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 From keithw at mit.edu Thu Dec 5 16:16:23 2013 From: keithw at mit.edu (Keith Winstein) Date: Thu, 5 Dec 2013 19:16:23 -0500 Subject: [e2e] Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> Message-ID: On Thu, Dec 5, 2013 at 6:22 PM, Andrew Mcgregor wrote: > By the way, fq_codel is not a combination of SFQ and CoDel, but something > else. The code deserves really close study to understand what is really > going on there, because there is a very interesting contribution in terms > of heuristically measuring whether flows are building queue in the qdisc or > not. Empirically, this works really well on user access links, including > LTE, by my own direct observation having run it on my home network's LTE > link for many months now. Andrew, I'd love to hear more about how you do this. Do you run fq_codel on the uplink or somehow also on the downlink? How do you prevent the LTE modem or baseband from buffering hundreds of kilobytes inside itself -- do you use an upstream traffic shaper (to prevent the LTE link from becoming the bottleneck) or something more sophisticated? Cheers, Keith From andrewmcgr at google.com Thu Dec 5 16:23:41 2013 From: andrewmcgr at google.com (Andrew Mcgregor) Date: Fri, 6 Dec 2013 11:23:41 +1100 Subject: [e2e] Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> Message-ID: I'm running a fairly standard OpenWRT QoS config, with ingress and egress shaping to prevent (too much) upstream queueing either in the network or in the modem; I get about 22Mbps symmetric performance on this particular setup, with reasonable latencies (I wouldn't be happy if I was FPS gaming, but it's fine for everything else). I don't remember the latency numbers off the top of my head. It's actually one of the better broadband connections available in Australia, it's just a shame about the cost. On 6 December 2013 11:16, Keith Winstein wrote: > On Thu, Dec 5, 2013 at 6:22 PM, Andrew Mcgregor wrote: > >> By the way, fq_codel is not a combination of SFQ and CoDel, but something >> else. The code deserves really close study to understand what is really >> going on there, because there is a very interesting contribution in terms >> of heuristically measuring whether flows are building queue in the qdisc >> or >> not. Empirically, this works really well on user access links, including >> LTE, by my own direct observation having run it on my home network's LTE >> link for many months now. > > > Andrew, I'd love to hear more about how you do this. Do you run fq_codel > on the uplink or somehow also on the downlink? How do you prevent the LTE > modem or baseband from buffering hundreds of kilobytes inside itself -- do > you use an upstream traffic shaper (to prevent the LTE link from becoming > the bottleneck) or something more sophisticated? > > Cheers, > Keith > -- Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 From detlef.bosau at web.de Fri Dec 6 03:02:47 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Fri, 06 Dec 2013 12:02:47 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: <930C8FF4-2469-4258-A6CA-882789182A79@csail.mit.edu> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <52A083F3.7000205@web.de> <930C8FF4-2469-4258-A6CA-882789182A79@csail.mit.edu> Message-ID: <52A1AED7.7040302@web.de> Hari, yes, you use traces obtained from somewhere. And when I remember TCP ex machina, which was mentioned by Keith, you run into the same fallacy again and again. In TCP ex machina you calculated ex post an optimal CC algorithm - that means you optimized a target function in some variables and found an optimum. Fine. And completely useless for the next concrete run of the protocols where the scenario my be completely different from the "obtained one". The same now. We can discuss the TCP RTO, which is basically a confidence interval which is expected to cover, say, 95% of observered RTT samples. Now you observe 10 Millions of TCP samples and calculate the RTO from these. And guess what is most likely to be found (by the Strong Law of Large Numbers). Of course, 95% of the observed RTT will stay in the "predicted" interval, because you have drawn "random numbers" by completely independent and identically distributed random variables obeying exactly the one distribution which is defined by your observations. And of course, the same formulae applied to _REAL_ observations will NOT hold. Because the _REAL_ observations are neither independent, nor identically distributed. This is always what is sometimes called "ratio ex post", sometimes "conclusion in the wrong direction", however this is mathematically wrong. And as you know from elementary physics (applied to Rayleigh Fading): You will NEVER be able to reproduce a mobile wireless scenario. Never. Under no circumstances. So it absolutely does not matter, whether you model doesn't match reality, or the "real world trace" - where the mobile phone was placed 5 centimetres away from its current location doesn't match the (current) reality. The error is always the same and you will, with "best QoS guarantees", fall into exactly the same pitfall and the same fallacy again and again. And although I don't remember the paper: I've read even the "trace story" in the context of adaptive multimedia flows over wireless networks years ago, the story is as old as wrong. Excuse my being upset. It's just a result of 14 years of frustration :-( Particularly as I well understand the problems. I made many of these mistakes myself and several times in my life (I'm aged 50 years, btw.), however: It is never a problem to go wrong. But it is always a problem to make the same error more than once. Detlef Am 05.12.2013 17:30, schrieb Hari Balakrishnan: > Detlef, > > May I suggest that you please look at the paper? The simulation experiments don't use a wireless link model but are obtained over packet delivery traces collected over multiple different cellular networks (in both directions). > > Hari > > On Dec 5, 2013, at 8:47 AM, Detlef Bosau wrote: > >> Keith, >> >> as long as we don't have compelling wireless link models in the ns-2, I >> don't think simulations in this area are useful. >> >> Particularly, quantitative results from those simulations are extremely >> questionable. >> >> -- >> ------------------------------------------------------------------ >> Detlef Bosau >> Galileistra?e 30 >> 70565 Stuttgart Tel.: +49 711 5208031 >> mobile: +49 172 6819937 >> skype: detlef.bosau >> ICQ: 566129673 >> detlef.bosau at web.de http://www.detlef-bosau.de >> -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From Jon.Crowcroft at cl.cam.ac.uk Fri Dec 6 04:49:37 2013 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Fri, 06 Dec 2013 12:49:37 +0000 Subject: [e2e] Codel and Wireless In-Reply-To: <52A1AED7.7040302@web.de> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <52A083F3.7000205@web.de> <930C8FF4-2469-4258-A6CA-882789182A79@csail.mit.edu> <52A1AED7.7040302@web.de> Message-ID: i think the variations in wireless properties over the length of their traes for their experiment probably covers most tof hte cases of variation you might see in future scenarios....there just isn't that much design space for the channel to vary over - they aren't trying to evolve/deisgn a channel condition predictor - just a reactive system that does betterer than others.... i think the Reverend Bayes has some things to observe in this space about a priori and a posteriori, although i might be talking out of my posterior when I say that In missive <52A1AED7.7040302 at web.de>, Detlef Bosau typed: >>Hari, >> >>yes, you use traces obtained from somewhere. And when I remember TCP ex >>machina, which was mentioned by Keith, >>you run into the same fallacy again and again. >> >>In TCP ex machina you calculated ex post an optimal CC algorithm - that >>means you optimized a target function in some variables and found an >>optimum. >> >>Fine. >> >>And completely useless for the next concrete run of the protocols where >>the scenario my be completely different from the "obtained one". >> >>The same now. >> >>We can discuss the TCP RTO, which is basically a confidence interval >>which is expected to cover, say, 95% of observered RTT samples. >> >>Now you observe 10 Millions of TCP samples and calculate the RTO from >>these. And guess what is most likely to be found (by the Strong Law of >>Large Numbers). >> >>Of course, 95% of the observed RTT will stay in the "predicted" >>interval, because you have drawn "random numbers" by completely >>independent and identically distributed random variables obeying exactly >>the one distribution which is defined by your observations. >> >>And of course, the same formulae applied to _REAL_ observations will NOT >>hold. Because the _REAL_ observations are neither independent, nor >>identically distributed. >> >>This is always what is sometimes called "ratio ex post", sometimes >>"conclusion in the wrong direction", however this is mathematically wrong. >> >> >>And as you know from elementary physics (applied to Rayleigh Fading): >>You will NEVER be able to reproduce a mobile wireless scenario. >>Never. Under no circumstances. So it absolutely does not matter, whether >>you model doesn't match reality, or the "real world trace" - where the >>mobile phone was placed 5 centimetres away from its current location >>doesn't match the (current) reality. >> >>The error is always the same and you will, with "best QoS guarantees", >>fall into exactly the same pitfall and the same fallacy again and again. >> >>And although I don't remember the paper: I've read even the "trace >>story" in the context of adaptive multimedia flows over wireless networks >>years ago, the story is as old as wrong. >> >>Excuse my being upset. It's just a result of 14 years of frustration :-( >> >>Particularly as I well understand the problems. I made many of these >>mistakes myself and several times in my life (I'm aged 50 years, btw.), >>however: >> >>It is never a problem to go wrong. But it is always a problem to make >>the same error more than once. >> >> >>Detlef >> >>Am 05.12.2013 17:30, schrieb Hari Balakrishnan: >>> Detlef, >>> >>> May I suggest that you please look at the paper? The simulation experiments don't use a wireless link model but are obtained over packet delivery traces collected over multiple different cellular networks (in both directions). >>> >>> Hari >>> >>> On Dec 5, 2013, at 8:47 AM, Detlef Bosau wrote: >>> >>>> Keith, >>>> >>>> as long as we don't have compelling wireless link models in the ns-2, I >>>> don't think simulations in this area are useful. >>>> >>>> Particularly, quantitative results from those simulations are extremely >>>> questionable. >>>> >>>> -- >>>> ------------------------------------------------------------------ >>>> Detlef Bosau >>>> Galileistra?e 30 >>>> 70565 Stuttgart Tel.: +49 711 5208031 >>>> mobile: +49 172 6819937 >>>> skype: detlef.bosau >>>> ICQ: 566129673 >>>> detlef.bosau at web.de http://www.detlef-bosau.de >>>> >> >> >>-- >>------------------------------------------------------------------ >>Detlef Bosau >>Galileistra?e 30 >>70565 Stuttgart Tel.: +49 711 5208031 >> mobile: +49 172 6819937 >> skype: detlef.bosau >> ICQ: 566129673 >>detlef.bosau at web.de http://www.detlef-bosau.de >> cheers jon From detlef.bosau at web.de Fri Dec 6 06:17:17 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Fri, 06 Dec 2013 15:17:17 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> Message-ID: <52A1DC6D.4000306@web.de> Am 06.12.2013 00:22, schrieb Andrew Mcgregor: > > Of course it isn't a measure of load, but that is not what CoDel is > trying to do with it, still less fq_codel. What these algorithms are > doing is effectively saying, we know that excess queueing delay (due > to TCP windowed flow control) is a problem in and of itself, so we > shall measure and control it directly. Why is queueing delay itself is a problem? >From a user's perspective (I take this from time to time), there are two concerns: a) Will the packet reach its destination at all? b) If it does, which time is needed for the travel? Did I miss someting? From detlef.bosau at web.de Fri Dec 6 16:03:46 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sat, 07 Dec 2013 01:03:46 +0100 Subject: [e2e] A "Railway Model." Re: Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <52A083F3.7000205@web.de> <930C8FF4-2469-4258-A6CA-882789182A79@csail.mit.edu> <52A1AED7.7040302@web.de> Message-ID: <52A265E2.2080907@web.de> Am 06.12.2013 13:49, schrieb Jon Crowcroft: > i think the variations in wireless properties > over the length of their traes for their experiment > probably covers most tof hte cases of variation you might > see in future scenarios....there just isn't that much > design space for the channel to vary over - they aren't trying to > evolve/deisgn a channel condition predictor - just a reactive system > that does betterer than others.... > > i think the Reverend Bayes has some things to observe in this space > about a priori and a posteriori, although i might be talking out of my > posterior when I say that And perhaps for simulations, this is perfectly fine. Nevertheless, although I'm writing much about wireless networks, I don't want to be a pure wireless guy. Actually, wireless networks cause some difficulties for TCP - when you run TCP along a single mobile link, some things are different from wirebound ones, e.g. the loss differentiation. However, this is not the core question of TCP congestion control. Perhaps, I'm a bit influenced by my residence, the famous city with water canons and without a train station (the water canons are used against those who want to keep the train station as written e.g. in the Washington Post,...), where it is quite common when you travel from one station to another ("Feuersee" to "Stadtmitte", a walk of about five minutes) that your train will pause several times: "Dear passengers, unfortunately the section in front of us is still occupied by another train, we will continue our trip in a few minutes." So, the railway is split up into sections. A train must not enter an occupied section. It has to wait for the section becoming available instead. Compared to TCP, this is EXACTLY, what we want. A TCP packet must not enter an occupied link, it has to wait for the link becoming available. And actually, it is not the link what becomes available - but the next hop, So a packet travels from hop to hop, in each case waiting for the next hop becoming available. There is no loss differentiation problem, there is no assessment of resources, there is no probing. Just a travel from section to section, and waiting for the next hop becoming available in each case. Just as if a packet were travelling from "Feuersee" to "Stadtmitte". This is just a thought. However, wouldn't this thought be at least interesting as a strategy for resource sharing and congestion avoidance? Yes, it would work on wireless networks. Yes, it would work on "long fat networks". And it would work on wire bound networks. And perhaps it could spare us some problems. Actually, that's what I had in mind, when I started this whole discussion. Is this thought completely weird? If not, where are the pitfalls? Pehaps, it makes sense to ask this question right to you, because it is not that unusual to read even unusual ideas in your posts ;-) From andrewmcgr at google.com Sat Dec 7 03:11:57 2013 From: andrewmcgr at google.com (Andrew Mcgregor) Date: Sat, 7 Dec 2013 22:11:57 +1100 Subject: [e2e] A "Railway Model." Re: Codel and Wireless In-Reply-To: <52A265E2.2080907@web.de> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <52A083F3.7000205@web.de> <930C8FF4-2469-4258-A6CA-882789182A79@csail.mit.edu> <52A1AED7.7040302@web.de> <52A265E2.2080907@web.de> Message-ID: This is hop-by-hop flow control, as implemented with 802.1 pause frames. It works, kind of. But there's a phenomenon called an 'incast pause cone', in which a single heavily loaded interface can pause an entire data center fabric. So it's not a panacea, in fact it has a whole menagerie of its own congestion issues. Infiniband also has hop-by-hop control, and has to be very carefully admission controlled to avoid these issues. The net effect is that you cannot use about 30% of the ostensible performance of any interface in the network, to avoid fabric deadlocks. Internet-style congestion control gets better utilisation, which is good if the links are expensive. On 7 December 2013 11:03, Detlef Bosau wrote: > Am 06.12.2013 13:49, schrieb Jon Crowcroft: > > i think the variations in wireless properties > > over the length of their traes for their experiment > > probably covers most tof hte cases of variation you might > > see in future scenarios....there just isn't that much > > design space for the channel to vary over - they aren't trying to > > evolve/deisgn a channel condition predictor - just a reactive system > > that does betterer than others.... > > > > i think the Reverend Bayes has some things to observe in this space > > about a priori and a posteriori, although i might be talking out of my > > posterior when I say that > > > > And perhaps for simulations, this is perfectly fine. > > Nevertheless, although I'm writing much about wireless networks, I don't > want to be a pure wireless guy. > > Actually, wireless networks cause some difficulties for TCP - when you > run TCP along a single mobile link, some things are different from > wirebound ones, e.g. the loss differentiation. > > However, this is not the core question of TCP congestion control. > > Perhaps, I'm a bit influenced by my residence, the famous city with > water canons and without a train station (the water canons are used > against those who want to keep the train station as written e.g. in the > Washington Post,...), where it is quite common when you travel from one > station to another ("Feuersee" to "Stadtmitte", a walk of about five > minutes) that your train will pause several times: "Dear passengers, > unfortunately the section in front of us is still occupied by another > train, we will continue our trip in a few minutes." So, the railway is > split up into sections. A train must not enter an occupied section. It > has to wait for the section becoming available instead. > > Compared to TCP, this is EXACTLY, what we want. A TCP packet must not > enter an occupied link, it has to wait for the link becoming available. > > And actually, it is not the link what becomes available - but the next > hop, So a packet travels from hop to hop, in each case waiting for the > next hop becoming available. There is no loss differentiation problem, > there is no assessment of resources, there is no probing. Just a travel > from section to section, and waiting for the next hop becoming available > in each case. > > Just as if a packet were travelling from "Feuersee" to "Stadtmitte". > > This is just a thought. However, wouldn't this thought be at least > interesting as a strategy for resource sharing and congestion avoidance? > > Yes, it would work on wireless networks. > Yes, it would work on "long fat networks". > And it would work on wire bound networks. > And perhaps it could spare us some problems. > > > Actually, that's what I had in mind, when I started this whole discussion. > > Is this thought completely weird? If not, where are the pitfalls? > > Pehaps, it makes sense to ask this question right to you, because it is > not that unusual to read even unusual ideas in your posts ;-) > > > > -- Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 From neil.davies at pnsol.com Sat Dec 7 04:12:04 2013 From: neil.davies at pnsol.com (Neil Davies) Date: Sat, 7 Dec 2013 12:12:04 +0000 Subject: [e2e] A "Railway Model." Re: Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <52A083F3.7000205@web.de> <930C8FF4-2469-4258-A6CA-882789182A79@csail.mit.edu> <52A1AED7.7040302@web.de> <52A265E2.2080907@web.de> Message-ID: <6F27AC2F-501C-4BDA-9D05-D9D0777514E8@pnsol.com> Actually, with worm-hole routed fabrics you can achieve better than 67% [1] for random paths with the right switching architecture. You are right, there are contention trees that form (spanning out from the bottleneck) in ALL such fabrics. You need about 1.41 (root 2) the edge capacity in the switching core to sustain all non-pathalogical traffic destination patterns at load[2] Its a system with two degrees of freedom - either you get delay or loss - your choice. We've found that the best approach is a polyservice one, not a monoservice one - push the trading space up the value chain, work with both loss and delay characteristics in a consistent framework.[3] Neil [1] Jones, Davies, Firth and Wright, The Network Designers Handbook, IOS press, 1997 ISBN 9051993905 [2] A Jomah, Instability in Switching Systems, PhD, Bristol 2000 [3] D Reeve, A New Blueprint for Network QoS, PhD, Kent , 2003 (Jon Crowcroft eternal examiner?) On 7 Dec 2013, at 11:11, Andrew Mcgregor wrote: > This is hop-by-hop flow control, as implemented with 802.1 pause frames. > It works, kind of. But there's a phenomenon called an 'incast pause > cone', in which a single heavily loaded interface can pause an entire data > center fabric. So it's not a panacea, in fact it has a whole menagerie of > its own congestion issues. Infiniband also has hop-by-hop control, and has > to be very carefully admission controlled to avoid these issues. The net > effect is that you cannot use about 30% of the ostensible performance of > any interface in the network, to avoid fabric deadlocks. Internet-style > congestion control gets better utilisation, which is good if the links are > expensive. > > > On 7 December 2013 11:03, Detlef Bosau wrote: > >> Am 06.12.2013 13:49, schrieb Jon Crowcroft: >>> i think the variations in wireless properties >>> over the length of their traes for their experiment >>> probably covers most tof hte cases of variation you might >>> see in future scenarios....there just isn't that much >>> design space for the channel to vary over - they aren't trying to >>> evolve/deisgn a channel condition predictor - just a reactive system >>> that does betterer than others.... >>> >>> i think the Reverend Bayes has some things to observe in this space >>> about a priori and a posteriori, although i might be talking out of my >>> posterior when I say that >> >> >> >> And perhaps for simulations, this is perfectly fine. >> >> Nevertheless, although I'm writing much about wireless networks, I don't >> want to be a pure wireless guy. >> >> Actually, wireless networks cause some difficulties for TCP - when you >> run TCP along a single mobile link, some things are different from >> wirebound ones, e.g. the loss differentiation. >> >> However, this is not the core question of TCP congestion control. >> >> Perhaps, I'm a bit influenced by my residence, the famous city with >> water canons and without a train station (the water canons are used >> against those who want to keep the train station as written e.g. in the >> Washington Post,...), where it is quite common when you travel from one >> station to another ("Feuersee" to "Stadtmitte", a walk of about five >> minutes) that your train will pause several times: "Dear passengers, >> unfortunately the section in front of us is still occupied by another >> train, we will continue our trip in a few minutes." So, the railway is >> split up into sections. A train must not enter an occupied section. It >> has to wait for the section becoming available instead. >> >> Compared to TCP, this is EXACTLY, what we want. A TCP packet must not >> enter an occupied link, it has to wait for the link becoming available. >> >> And actually, it is not the link what becomes available - but the next >> hop, So a packet travels from hop to hop, in each case waiting for the >> next hop becoming available. There is no loss differentiation problem, >> there is no assessment of resources, there is no probing. Just a travel >> from section to section, and waiting for the next hop becoming available >> in each case. >> >> Just as if a packet were travelling from "Feuersee" to "Stadtmitte". >> >> This is just a thought. However, wouldn't this thought be at least >> interesting as a strategy for resource sharing and congestion avoidance? >> >> Yes, it would work on wireless networks. >> Yes, it would work on "long fat networks". >> And it would work on wire bound networks. >> And perhaps it could spare us some problems. >> >> >> Actually, that's what I had in mind, when I started this whole discussion. >> >> Is this thought completely weird? If not, where are the pitfalls? >> >> Pehaps, it makes sense to ask this question right to you, because it is >> not that unusual to read even unusual ideas in your posts ;-) >> >> >> >> > > > -- > Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 8143 7128 From detlef.bosau at web.de Sat Dec 7 05:36:14 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sat, 07 Dec 2013 14:36:14 +0100 Subject: [e2e] A "Railway Model." Re: Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <52A083F3.7000205@web.de> <930C8FF4-2469-4258-A6CA-882789182A79@csail.mit.edu> <52A1AED7.7040302@web.de> <52A265E2.2080907@web.de> Message-ID: <52A3244E.1060900@web.de> Am 07.12.2013 12:11, schrieb Andrew Mcgregor: > This is hop-by-hop flow control, Yes, it _is_ hop by hop flow control. > as implemented with 802.1 pause frames. 802.1? I think, you mean Ethernet. > It works, kind of. But there's a phenomenon called an 'incast pause > cone', This is surely not the only problem with Ethernet flow control. (Do you have a reference for the incast pause cone problem?) I did not discuss the details how these can be overcome. (And as far as I see: They can be overcome.) -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From detlef.bosau at web.de Sat Dec 7 07:08:26 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sat, 07 Dec 2013 16:08:26 +0100 Subject: [e2e] A "Railway Model." Re: Codel and Wireless In-Reply-To: <6F27AC2F-501C-4BDA-9D05-D9D0777514E8@pnsol.com> References: <52963EF4.7000908@web.de> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <52A083F3.7000205@web.de> <930C8FF4-2469-4258-A6CA-882789182A79@csail.mit.edu> <52A1AED7.7040302@web.de> <52A265E2.2080907@web.de> <6F27AC2F-501C-4BDA-9D05-D9D0777514E8@pnsol.com> Message-ID: <52A339EA.7000907@web.de> Am 07.12.2013 13:12, schrieb Neil Davies: > Actually, with worm-hole routed fabrics you can achieve better than 67% [1] for random paths with the right switching architecture. > Nevertheless, Andrew mixed up the layers. I'm talking about TCP or equivalent protocols. Not about Layer 2. And neither about Ethernet pause frames. So, perhaps Andrew wants to have a second look to what I suggested. (I gave it more than two thoughts before I wrote it, so one could give it two attempts in reading it ;-)) -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From Jon.Crowcroft at cl.cam.ac.uk Sat Dec 7 08:04:00 2013 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Sat, 07 Dec 2013 16:04:00 +0000 Subject: [e2e] A "Railway Model." Re: Codel and Wireless In-Reply-To: <52A339EA.7000907@web.de> References: <52963EF4.7000908@web.de> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <52A083F3.7000205@web.de> <930C8FF4-2469-4258-A6CA-882789182A79@csail.mit.edu> <52A1AED7.7040302@web.de> <52A265E2.2080907@web.de> <6F27AC2F-501C-4BDA-9D05-D9D0777514E8@pnsol.com> <52A339EA.7000907@web.de> Message-ID: you might want to read about how people use layer two congestion signaling from L2 (only) switches to give feedback to TCP which then uses a distributed scheduler to avoid the incast problem alluded to... yes, gasp, layer violation - but it works. so engineers like it In missive <52A339EA.7000907 at web.de>, Detlef Bosau typed: >>Am 07.12.2013 13:12, schrieb Neil Davies: >>> Actually, with worm-hole routed fabrics you can achieve better than 67% [1] for random paths with the right switching architecture. >>> >>Nevertheless, Andrew mixed up the layers. >> >>I'm talking about TCP or equivalent protocols. Not about Layer 2. And >>neither about Ethernet pause frames. >>So, perhaps Andrew wants to have a second look to what I suggested. >>(I gave it more than two thoughts before I wrote it, so one could give >>it two attempts in reading it ;-)) >> >>-- >>------------------------------------------------------------------ >>Detlef Bosau >>Galileistra?e 30 >>70565 Stuttgart Tel.: +49 711 5208031 >> mobile: +49 172 6819937 >> skype: detlef.bosau >> ICQ: 566129673 >>detlef.bosau at web.de http://www.detlef-bosau.de >> cheers jon From detlef.bosau at web.de Sat Dec 7 08:17:10 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sat, 07 Dec 2013 17:17:10 +0100 Subject: [e2e] A "Railway Model." Re: Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <52A083F3.7000205@web.de> <930C8FF4-2469-4258-A6CA-882789182A79@csail.mit.edu> <52A1AED7.7040302@web.de> <52A265E2.2080907@web.de> <6F27AC2F-501C-4BDA-9D05-D9D0777514E8@pnsol.com> <52A339EA.7000907@web.de> Message-ID: <52A34A06.6050304@web.de> Am 07.12.2013 17:04, schrieb Jon Crowcroft: > you might want to read about how people use layer two congestion > signaling from L2 (only) switches to give feedback to TCP which then > uses a distributed scheduler to avoid the incast problem alluded to... > > yes, gasp, layer violation - but it works. so engineers like it I don't mind layer violation. However, a L3 resource problem is slightly different from L2 flow control. Only to mention two issues: 1.: Passing L2 congestion information to L3 is likely to end up in source quench or the like. Which sources are throttled? To which degree? What happens to traffic which is not associated to any flow? 2.: The very first problem met when you want to employ L2 flow control for upper layers is Head of Line Blocking. To my understanding, this is THE very reason why the use of L2 congestion control was silently abandoned in the turn from Cerf's catenet to RFC 791 -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From Jon.Crowcroft at cl.cam.ac.uk Sat Dec 7 08:28:46 2013 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Sat, 07 Dec 2013 16:28:46 +0000 Subject: [e2e] A "Railway Model." Re: Codel and Wireless In-Reply-To: <52A34A06.6050304@web.de> References: <52963EF4.7000908@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <52A083F3.7000205@web.de> <930C8FF4-2469-4258-A6CA-882789182A79@csail.mit.edu> <52A1AED7.7040302@web.de> <52A265E2.2080907@web.de> <6F27AC2F-501C-4BDA-9D05-D9D0777514E8@pnsol.com> <52A339EA.7000907@web.de> <52A34A06.6050304@web.de> Message-ID: you have to read the literature - the data center tcp variants that avoid incast do so by passing l2 feedback (which is part of the ethernet spec - you can send a control frame which indicates a new inter-frame gap) through the IP layer and on to TCP sources (actually, it doesn't HAVE to work that way- you can just have per-flow queues in the soruce machine and apply the dynamic inter-frame gap/delay differently to different queues.... so there's no layer 2 congestion +control+ - just queue feedback that is from l2 and a mechanism to pace packets... actually, you could do this in DCF in wifi too:) In missive <52A34A06.6050304 at web.de>, Detlef Bosau typed: >>Am 07.12.2013 17:04, schrieb Jon Crowcroft: >>> you might want to read about how people use layer two congestion >>> signaling from L2 (only) switches to give feedback to TCP which then >>> uses a distributed scheduler to avoid the incast problem alluded to... >>> >>> yes, gasp, layer violation - but it works. so engineers like it >> >>I don't mind layer violation. >> >>However, a L3 resource problem is slightly different from L2 flow control. >> >>Only to mention two issues: >> >>1.: Passing L2 congestion information to L3 is likely to end up in >>source quench or the like. >> >>Which sources are throttled? To which degree? What happens to traffic >>which is not associated to any flow? >> >>2.: The very first problem met when you want to employ L2 flow control >>for upper layers is Head of Line Blocking. To my understanding, this is >>THE very reason why the use of L2 congestion control was silently >>abandoned in the turn from Cerf's catenet to RFC 791 >> >> >>-- >>------------------------------------------------------------------ >>Detlef Bosau >>Galileistra?e 30 >>70565 Stuttgart Tel.: +49 711 5208031 >> mobile: +49 172 6819937 >> skype: detlef.bosau >> ICQ: 566129673 >>detlef.bosau at web.de http://www.detlef-bosau.de >> cheers jon From detlef.bosau at web.de Sat Dec 7 14:40:28 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sat, 07 Dec 2013 23:40:28 +0100 Subject: [e2e] A "Railway Model." Re: Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <1386112241.18127.YahooMailNeo@web141606.mail.bf1.yahoo.com> <52A083F3.7000205@web.de> <930C8FF4-2469-4258-A6CA-882789182A79@csail.mit.edu> <52A1AED7.7040302@web.de> <52A265E2.2080907@web.de> <6F27AC2F-501C-4BDA-9D05-D9D0777514E8@pnsol.com> <52A339EA.7000907@web.de> <52A34A06.6050304@web.de> Message-ID: <52A3A3DC.4060007@web.de> Am 07.12.2013 17:28, schrieb Jon Crowcroft: > you have to read the literature - the data center tcp variants that > avoid incast do so by passing l2 feedback (which is part of the > ethernet spec - you can send a control frame which indicates a new > inter-frame gap) through the IP layer and on to TCP sources (actually, > it doesn't HAVE to work that way- you can just have per-flow queues > in the soruce machine and apply the dynamic inter-frame gap/delay > differently to different queues.... You should read the RFC about Nagle's algorithm. That's to these old wife's tales with the interframe gap. (BTW: I didn't write a word about interframe gaps. Guess why....) > > so there's no layer 2 congestion +control+ - Of course there is. It just has another name: Media access control. And in dedicated media, there is no need for congestion control because there is no media capacity to be shared among several senders. > just queue feedback that > is from l2 and a mechanism to pace packets... Pacing is a nice thing. Particularly if you pace with the right rate..... Just a note: TCP is an asynchronous protocol. So, there is no real need for any kind of pacing, particularly not on an end to end basis. (Exactly this end to end pacing AKA self clocking is one of the central fallacies of TCP congestion control. Perhaps the most important one. This is just the very point where we never understood, that the Internet is a packet switched network and not a reinvented telephone system with a keyboard instead of a dial plate and routers which work as strowger gears with noise insulation.) > > actually, you could do this in DCF in wifi too:) Yes. Indeed. Wifi has a MAC scheme. (What else is DCF than congestion control? It even uses inter frame gaps!) (Perhaps, we should at least talk about Token Ring here, because it is really a bit funny to talk about the allocation for media capacity on a shared medium which can serve a maximum of ONE packet at any time. And just as a gedankenexperiment: Imagine two nodes interconnected by an Ethernet cabling: What's the use of VJCC then? Basically none. The best idea is to turn it of - to avoid being bothered by this stuff.) (Yes, I know the capture effect, so CSMA/CD isn't fair. What did Wes Crusher say in StarTrek? "Life isn't always fair.") In my opinion, it is basically a good idea to leave fairness to schedulers and not to self scheduling and other sophisticated instances of "swarm intelligence". And yes, that makes use of technological progress: At the garden hose's times, there was no alternative to CSMA/CD. ("There is no alternative", that was a long time ago. IIRC at the time, when chancellor Merkel created the garden of Eden.) Meanwhile (by the time of the stone age) switches have been invented - and now, fairness is left to schedulers. So, shared media could be left to courses on network history. (Yes, this is no politcial blog here. But as many of you know, meanwhile chancellor Merkel has admitted to be an Internet newbie. No one had the slightest idea of this before...) -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From detlef.bosau at web.de Sat Dec 14 13:11:49 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sat, 14 Dec 2013 22:11:49 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: <52A1DC6D.4000306@web.de> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> <52A1DC6D.4000306@web.de> Message-ID: <52ACC995.3070104@web.de> Am 06.12.2013 15:17, schrieb Detlef Bosau: > Why is queueing delay itself is a problem? >From a user's perspective > (I take this from time to time), there are two concerns: a) Will the > packet reach its destination at all? b) If it does, which time is > needed for the travel? Did I miss someting? I'm a bit curious, why I got no answer to my question. -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From andrewmcgr at google.com Sat Dec 14 17:21:43 2013 From: andrewmcgr at google.com (Andrew Mcgregor) Date: Sun, 15 Dec 2013 12:21:43 +1100 Subject: [e2e] Codel and Wireless In-Reply-To: <52ACC995.3070104@web.de> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> <52A1DC6D.4000306@web.de> <52ACC995.3070104@web.de> Message-ID: I've been out of the office and not thinking about how to explain further. But I shall. On 15 Dec 2013 08:30, "Detlef Bosau" wrote: > Am 06.12.2013 15:17, schrieb Detlef Bosau: > > Why is queueing delay itself is a problem? >From a user's perspective > > (I take this from time to time), there are two concerns: a) Will the > > packet reach its destination at all? b) If it does, which time is > > needed for the travel? Did I miss someting? > > I'm a bit curious, why I got no answer to my question. > > -- > ------------------------------------------------------------------ > Detlef Bosau > Galileistra?e 30 > 70565 Stuttgart Tel.: +49 711 5208031 > mobile: +49 172 6819937 > skype: detlef.bosau > ICQ: 566129673 > detlef.bosau at web.de http://www.detlef-bosau.de > > From detlef.bosau at web.de Sat Dec 14 17:28:31 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sun, 15 Dec 2013 02:28:31 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: <52ACC995.3070104@web.de> References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> <52A1DC6D.4000306@web.de> <52ACC995.3070104@web.de> Message-ID: <52AD05BF.3040309@web.de> Am 14.12.2013 22:11, schrieb Detlef Bosau: > Am 06.12.2013 15:17, schrieb Detlef Bosau: >> Why is queueing delay itself is a problem? >From a user's perspective >> (I take this from time to time), there are two concerns: a) Will the >> packet reach its destination at all? b) If it does, which time is >> needed for the travel? Did I miss someting? > I'm a bit curious, why I got no answer to my question. > In addition: On the one hand, we all praise the "end to end principle". On the other hand, we know that VJCC (and its flavours) runs into severe problems with mobile links and LFN as well, which makes it inappropriate for generic use, and while we repeat again and again that only the end nodes can make relevant decisions we subvert the end to end principle e.g. with any kind of more or less sophisticated queue management approaches, one of which is CoDel, middle boxes and link specific technologies. Hence, e2e is likewise a credo - and obsolete as well. In a sense, we're odds with ourselves here. And in the same sense, we run into precisely the logjam, Ford and Iyengar complained about in 2009 and VJ himself complained about 2006. We see limits in our congestion handling - and try to overcome these by repeating the same approaches again and again for about 20 years now. While at least Ford & Iyengar made steps into a possible direction. Perhaps, F&I did not provide a silver-bullet solution. However, they asked questions. (And as I'm told, Lt. Worf from StarTrek TNG uttered in the episode "The rightful heir" the words: "Questions are the beginning of wisdom".) And perhaps, one important question is: Is the e2e principle the correct approach to prevent congestion control and to provide resource management (which is basically the same) in packet switching networks? -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From naeem.khademi at gmail.com Sat Dec 14 21:35:04 2013 From: naeem.khademi at gmail.com (Naeem Khademi) Date: Sun, 15 Dec 2013 06:35:04 +0100 Subject: [e2e] What is a good burst? -- AQM evaluation guidelines Message-ID: Hi all I'm not sure if this has already been covered in any of the other threads, but looking at http://www.ietf.org/proceedings/88/slides/slides-88-aqm-5.pdfand draft-ietf-aqm-recommendation-00, the question remains: "what is a good burst (size) that AQMs should allow?" and/or "how an AQM can have a notion of the right burst size?". and how "naturally-occuring bursts" mentioned in draft-ietf-aqm-recommendation-00 can be defined? Regards, Naeem From detlef.bosau at web.de Sun Dec 15 09:37:56 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sun, 15 Dec 2013 18:37:56 +0100 Subject: [e2e] What is a good burst? -- AQM evaluation guidelines In-Reply-To: References: Message-ID: <52ADE8F4.7080604@web.de> Admittedly, my position on this matter radically changed during the last years. However: TCP is an asynchronous protocol. So, if there are no other reasons to avoid bursts, I don't see a problem in bursty traffic. Am 15.12.2013 06:35, schrieb Naeem Khademi: > Hi all > > I'm not sure if this has already been covered in any of the other threads, > but looking at http://www.ietf.org/proceedings/88/slides/slides-88-aqm-5.pdfand > draft-ietf-aqm-recommendation-00, the question remains: "what is a > good > burst (size) that AQMs should allow?" and/or "how an AQM can have a notion > of the right burst size?". > > and how "naturally-occuring bursts" mentioned in > draft-ietf-aqm-recommendation-00 > can be defined? > > > > Regards, > Naeem -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From naeem.khademi at gmail.com Sun Dec 15 10:47:54 2013 From: naeem.khademi at gmail.com (Naeem Khademi) Date: Sun, 15 Dec 2013 19:47:54 +0100 Subject: [e2e] What is a good burst? -- AQM evaluation guidelines In-Reply-To: <52ADE8F4.7080604@web.de> References: <52ADE8F4.7080604@web.de> Message-ID: Hi Detlaf My question was rather what types of bursts, AQMs should or shouldn't allow? -- e.g. when designing my AQM should I care about 64K TSO-generated bursts to safely pass without dropping or not? Does the answer also apply to the burst sizes typical of multimedia traffic, etc.? if the answer is "yes", should an AQM design be actively aware of what application layer does in terms of sending bursty traffic or not? and to what extent if yes? Regards, Naeem On Sun, Dec 15, 2013 at 6:37 PM, Detlef Bosau wrote: > Admittedly, my position on this matter radically changed during the last > years. > > However: TCP is an asynchronous protocol. So, if there are no other > reasons to avoid bursts, I don't see a problem > in bursty traffic. > > Am 15.12.2013 06:35, schrieb Naeem Khademi: > > Hi all > > > > I'm not sure if this has already been covered in any of the other > threads, > > but looking at > http://www.ietf.org/proceedings/88/slides/slides-88-aqm-5.pdfand > > draft-ietf-aqm-recommendation-00, the question remains: "what is a > > good > > burst (size) that AQMs should allow?" and/or "how an AQM can have a > notion > > of the right burst size?". > > > > and how "naturally-occuring bursts" mentioned in > > draft-ietf-aqm-recommendation-00 > > can be defined? > > > > > > > > Regards, > > Naeem > > > -- > ------------------------------------------------------------------ > Detlef Bosau > Galileistra?e 30 > 70565 Stuttgart Tel.: +49 711 5208031 > mobile: +49 172 6819937 > skype: detlef.bosau > ICQ: 566129673 > detlef.bosau at web.de http://www.detlef-bosau.de > > From detlef.bosau at web.de Sun Dec 15 11:29:15 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sun, 15 Dec 2013 20:29:15 +0100 Subject: [e2e] What is a good burst? -- AQM evaluation guidelines In-Reply-To: References: <52ADE8F4.7080604@web.de> Message-ID: <52AE030B.3090401@web.de> Am 15.12.2013 19:47, schrieb Naeem Khademi: > Hi Detlaf > > My question was rather what types of bursts, AQMs should or shouldn't > allow? -- e.g. when designing my AQM should I care about 64K > TSO-generated bursts to safely pass without dropping or not What do you mean by TSO? Time Sharing Option by IBM? In the Internet, there is never any guarantee that packets aren't dropped. -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From naeem.khademi at gmail.com Sun Dec 15 11:41:42 2013 From: naeem.khademi at gmail.com (Naeem Khademi) Date: Sun, 15 Dec 2013 20:41:42 +0100 Subject: [e2e] What is a good burst? -- AQM evaluation guidelines In-Reply-To: <52AE030B.3090401@web.de> References: <52ADE8F4.7080604@web.de> <52AE030B.3090401@web.de> Message-ID: TCP Segmentation Offload. Naeem On Sun, Dec 15, 2013 at 8:29 PM, Detlef Bosau wrote: > Am 15.12.2013 19:47, schrieb Naeem Khademi: > > Hi Detlaf > > > > My question was rather what types of bursts, AQMs should or shouldn't > > allow? -- e.g. when designing my AQM should I care about 64K > > TSO-generated bursts to safely pass without dropping or not > > What do you mean by TSO? Time Sharing Option by IBM? > > > In the Internet, there is never any guarantee that packets aren't dropped. > > -- > ------------------------------------------------------------------ > Detlef Bosau > Galileistra?e 30 > 70565 Stuttgart Tel.: +49 711 5208031 > mobile: +49 172 6819937 > skype: detlef.bosau > ICQ: 566129673 > detlef.bosau at web.de http://www.detlef-bosau.de > > From detlef.bosau at web.de Sun Dec 15 11:54:45 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sun, 15 Dec 2013 20:54:45 +0100 Subject: [e2e] What is a good burst? -- AQM evaluation guidelines In-Reply-To: References: <52ADE8F4.7080604@web.de> <52AE030B.3090401@web.de> Message-ID: <52AE0905.2030408@web.de> Am 15.12.2013 20:41, schrieb Naeem Khademi: > TCP Segmentation Offload. > > Naeem O.k. Does not change my position. However, the question is where you place TCP congestion control then. In the end node's OS? Or in the the network adaptor? -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From naeem.khademi at gmail.com Sun Dec 15 12:03:09 2013 From: naeem.khademi at gmail.com (Naeem Khademi) Date: Sun, 15 Dec 2013 21:03:09 +0100 Subject: [e2e] What is a good burst? -- AQM evaluation guidelines In-Reply-To: <52AE0905.2030408@web.de> References: <52ADE8F4.7080604@web.de> <52AE030B.3090401@web.de> <52AE0905.2030408@web.de> Message-ID: On Sun, Dec 15, 2013 at 8:54 PM, Detlef Bosau wrote: > Am 15.12.2013 20:41, schrieb Naeem Khademi: > > TCP Segmentation Offload. > > Naeem > > > O.k. > > Does not change my position. > OK. > > However, the question is where you place TCP congestion control then. In > the end node's OS? Or in the the network adaptor? > This is not relevant to my question. I asked about what AQMs should do with it. Naeem > -- > ------------------------------------------------------------------ > Detlef Bosau > Galileistra?e 30 > 70565 Stuttgart Tel.: +49 711 5208031 > mobile: +49 172 6819937 > skype: detlef.bosau > ICQ: 566129673detlef.bosau at web.de http://www.detlef-bosau.de > > From detlef.bosau at web.de Sun Dec 15 12:59:59 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sun, 15 Dec 2013 21:59:59 +0100 Subject: [e2e] What is a good burst? -- AQM evaluation guidelines In-Reply-To: References: <52ADE8F4.7080604@web.de> <52AE030B.3090401@web.de> <52AE0905.2030408@web.de> Message-ID: <52AE184F.6090301@web.de> Am 15.12.2013 21:03, schrieb Naeem Khademi: > > On Sun, Dec 15, 2013 at 8:54 PM, Detlef Bosau > wrote: > > Am 15.12.2013 20:41, schrieb Naeem Khademi: >> TCP Segmentation Offload. >> >> Naeem > > O.k. > > Does not change my position. > > > OK. > > > > However, the question is where you place TCP congestion control > then. In the end node's OS? Or in the the network adaptor? > > > > This is not relevant to my question. I asked about what AQMs should do > with it. O.k. However: It's extremely interesting, that a pure "end to end task" is gradually being transferred to entities along the path, be it AQM, the introduction of middleboxes or whatever. -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From fred at cisco.com Sun Dec 15 13:42:12 2013 From: fred at cisco.com (Fred Baker (fred)) Date: Sun, 15 Dec 2013 21:42:12 +0000 Subject: [e2e] [aqm] What is a good burst? -- AQM evaluation guidelines In-Reply-To: References: Message-ID: <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com> On Dec 14, 2013, at 9:35 PM, Naeem Khademi wrote: > Hi all > > I'm not sure if this has already been covered in any of the other threads, but looking at http://www.ietf.org/proceedings/88/slides/slides-88-aqm-5.pdf and draft-ietf-aqm-recommendation-00, the question remains: "what is a good burst (size) that AQMs should allow?" and/or "how an AQM can have a notion of the right burst size?". > > and how "naturally-occuring bursts" mentioned in draft-ietf-aqm-recommendation-00 can be defined? Imagine, if you will, that you have a host and a network in front of it including a first hop switch or router.The host gas a TCP offload engine, which is a device that accepts a large chunk of data and sends as much of it as it has permission to send as quickly as it can. The host has, for sake of argument, a 10 MBPS interface, and everything else in the network has interfaces whose rate are measured in gigabits. The host gives its TSO one chunk of data, so that can't be called a "burst" - it's one message. The TSO sends data as quickly as it can, but presumably does little more than keep the transmission system operating without a pause; while it might queue up 45 messages at a crack, there is no requirement that it do so, so the term "burst" there doesn't have a lot of meaning. And as the data moves through the network, the rate of the particular session is absolutely lost in the available capacity. So a burst, in the sense of the definition, never happens. Now, repeat the experiment. However, in this case the host as a gig-E interface, and the next interface that its router uses is 10 or 100 MBPS. The host and its TSO, and for that matter the router, do exactly the same thing. As perceived by the router, data is arriving much more quickly than it is leaving, resulting in a temporarily deep queue. If the propagation delay through the remainder of the network and the destination host are appropriate, acknowledgements could arrive at the TSO, soliciting new transmissions, before that queue empties. In that case, it is very possible that the queue remains full for a period of time. This network event could last for quite some time. The second is clearly a burst, according to the definition, and I would argue that it is naturally occurring. I imagine you have heard Van and/or Kathy talk about "good queue" vs "bad queue". "Good queue" keeps enough traffic in it to fully utilize its egress. "Bad queue" also does so, but does so in a manner that also materially increases measured latency. This difference is what is behind my comment on the objective of a congestion management algorithm (such as TCP's but not limited to it) that its objective is to keep the amount of data outstanding large enough to maximize its transmission rate through the network, but not so large as to materially increase measured latency or probability of loss. I would argue that this concept of "Good Queue" is directly relevant to the concept of an acceptable burst size. In the first transmission in a session, the sender has no information about what it will experience, so it behoves it to behave in a manner that is unlikely to create a significant amount of "bad queue" - conservatively. But it by definition has no numbers by which to quantify that. Hence, we make recommendations about the initial window size. After that, I would argue that it should continue to behave in a manner that doesn't led to "bad queue", but is free to operate in any manner that seeks to keep the amount of data outstanding large enough to maximize its transmission rate through the network, but not so large as to materially increase measured latency or probability of loss. At the point that it sends data in a manner that creates a sustained queue, it has exceeded what would be considered a useful burst size. From fred at cisco.com Sun Dec 15 23:34:52 2013 From: fred at cisco.com (Fred Baker (fred)) Date: Mon, 16 Dec 2013 07:34:52 +0000 Subject: [e2e] [aqm] What is a good burst? -- AQM evaluation guidelines In-Reply-To: <201312152257.rBFMveZO022075@bagheera.jungle.bt.co.uk> References: <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com> <201312152257.rBFMveZO022075@bagheera.jungle.bt.co.uk> Message-ID: <80CEFF86-8687-4266-B87C-8FB5B894D4DF@cisco.com> On Dec 15, 2013, at 2:57 PM, Bob Briscoe wrote: > Fred, > > Jonathan Morton, Michael Scharf & I took Naeem's question to mean "What should an AQM assume the size of a good burst is?" whereas I think you and David C-B took the question to mean "What should an end-system take the size of a good burst to be?". I can't comment on what he means. I took the question as "what should a system that is in receipt of what it might consider a 'burst', and more especially a 'good burst', to be?" I don't know that a sending transport (which is to be distinguished from the queueing arrangement in that same system) or a receiving system *has* a definition of a "good" or "bad" burst. The one is sending data, which in the context of y two examples might be a good or bad idea, and the other is receiving it. From the receiver's perspective, the data either arrived or it didn't; if it arrived, there is no real argument for not delivering it to its application... From detlef.bosau at web.de Mon Dec 16 03:55:29 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Mon, 16 Dec 2013 12:55:29 +0100 Subject: [e2e] [aqm] What is a good burst? -- AQM evaluation guidelines In-Reply-To: <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com> References: <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com> Message-ID: <52AEEA31.6090107@web.de> Am 15.12.2013 22:42, schrieb Fred Baker (fred): > The second is clearly a burst, according to the definition, and I would argue that it is naturally occurring. I imagine you have heard Van and/or Kathy talk about "good queue" vs "bad queue". "Good queue" keeps enough traffic in it to fully utilize its egress. "Bad queue" also does so, but does so in a manner that also materially increases measured latency. This difference is what is behind my comment on the objective of a congestion management algorithm (such as TCP's but not limited to it) that its objective is to keep the amount of data outstanding large enough to maximize its transmission rate through the network, but not so large as to materially increase measured latency or probability of loss. I would argue that this concept of "Good Queue" is directly relevant to the concept of an acceptable burst size. In the first transmission in a session, the sender has no information about what it will experience, so it behoves it to behave in a manner that is unlikely to create a significant amount of "bad queue" - conservatively. But it by definition has no numbers by which to quantify that. Hence, we make recommendations about the initial window size. After that, I would argue that it should continue to behave in a manner that doesn't led to "bad queue", but is free to operate in any manner that seeks to keep the amount of data outstanding large enough to maximize its transmission rate through the network, but not so large as to materially increase measured latency or probability of loss. At the point that it sends data in a manner that creates a sustained queue, it has exceeded what would be considered a useful burst size. Isn't this a perfect argument for doing congestion control locally and not on the end nodes? An end node has absolutely no idea of good queues and bad queues, in addition window halving as congestion response neither sees this difference and empties good queues in the same way as bad ones. It may have sound extremely hard, when I called the end to end principle a "fallacy" some days ago, but what you write here, supports exactly that. -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From detlef.bosau at web.de Mon Dec 16 04:18:10 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Mon, 16 Dec 2013 13:18:10 +0100 Subject: [e2e] [aqm] What is a good burst? -- AQM evaluation guidelines In-Reply-To: <80CEFF86-8687-4266-B87C-8FB5B894D4DF@cisco.com> References: <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com> <201312152257.rBFMveZO022075@bagheera.jungle.bt.co.uk> <80CEFF86-8687-4266-B87C-8FB5B894D4DF@cisco.com> Message-ID: <52AEEF82.7040105@web.de> Am 16.12.2013 08:34, schrieb Fred Baker (fred): > On Dec 15, 2013, at 2:57 PM, Bob Briscoe > wrote: > >> Fred, >> >> Jonathan Morton, Michael Scharf & I took Naeem's question to mean "What should an AQM assume the size of a good burst is?" whereas I think you and David C-B took the question to mean "What should an end-system take the size of a good burst to be?". > I can't comment on what he means. I took the question as "what should a system that is in receipt of what it might consider a 'burst', and more especially a 'good burst', to be?" > > I don't know that a sending transport (which is to be distinguished from the queueing arrangement in that same system) or a receiving system *has* a definition of a "good" or "bad" burst. The one is sending data, which in the context of y two examples might be a good or bad idea, and the other is receiving it. From the receiver's perspective, the data either arrived or it didn't; if it arrived, there is no real argument for not delivering it to its application... As I stated ago, the whole AQM discussion of "good" and "bad" queues emphases, that congestion control is a local thing because congestion is a local thing. Some days ago, Dave Reed wrote that congestion control works fine when the data rates do not vary that much. Dear colleagues, the data rates of network links being actually in use varies on a range of 10 orders of magnitude. And this range may become larger. Some people try to manage this with sophisticated mechanisms on the end points, some others think the end points should be facilitated by AQM boxes who do sophisticated dropping. Dropping a packet is always wrong. Period. It's the networks job to deliver the packet, not to drop it. So dropping packets as a means of congestion control does not solve a problem but introduces a second one. There is not only a problem of network overload but a problem with a lossy network. In addition: The congestion collapses in the late 80s were, synonymously, retransmission collapses. The real problem was not that packets were dropped. The problem was that the network's capacity was insufficient to handle the retransmissions. Any packet drop is a potential retransmission. Our problem is that we abuse drop as a means of signalling. When an application/node/whatever offers to much load to the network, we should have this application/node/whatever offer less node to the network. And not drop the packets and hope the application will behave accordingly. It's like bringing up a child. Either you teach your child to make up his room. Or you will make up your child's room for the rest of your life. Detlef -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From naeem.khademi at gmail.com Mon Dec 16 05:47:34 2013 From: naeem.khademi at gmail.com (Naeem Khademi) Date: Mon, 16 Dec 2013 14:47:34 +0100 Subject: [e2e] [aqm] What is a good burst? -- AQM evaluation guidelines In-Reply-To: <80CEFF86-8687-4266-B87C-8FB5B894D4DF@cisco.com> References: <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com> <201312152257.rBFMveZO022075@bagheera.jungle.bt.co.uk> <80CEFF86-8687-4266-B87C-8FB5B894D4DF@cisco.com> Message-ID: Bob, Fred and all I'll copy/paste the question here again: "what is a good burst (size) that AQMs should allow?" and/or "how an AQM can have a notion of the right burst size?" So, obviously, as Bob mentioned, I'm concerned about what AQMs should or shouldn't do. The mission of dealing with packet bursts in addition to the task of keeping the standing queue very low or minimal is part of an "AQM evaluation criteria" I envision. While I do agree with all Fred's remarks, I'm more concerned to have an answer for this, for where AQMs might get deployed. An example: when designing my AQM X should I care about 64K TSO-generated bursts to safely pass without dropping or not? Does the answer (whatever it is) also apply to the burst sizes typical of multimedia traffic, etc.? if the answer is "yes", should an AQM design be actively aware of what application layer does in terms of sending bursty traffic or not? and to what extent if yes? Regards, Naeem On Mon, Dec 16, 2013 at 8:34 AM, Fred Baker (fred) wrote: > > On Dec 15, 2013, at 2:57 PM, Bob Briscoe > wrote: > > > Fred, > > > > Jonathan Morton, Michael Scharf & I took Naeem's question to mean "What > should an AQM assume the size of a good burst is?" whereas I think you and > David C-B took the question to mean "What should an end-system take the > size of a good burst to be?". > > I can't comment on what he means. I took the question as "what should a > system that is in receipt of what it might consider a 'burst', and more > especially a 'good burst', to be?" > > I don't know that a sending transport (which is to be distinguished from > the queueing arrangement in that same system) or a receiving system *has* a > definition of a "good" or "bad" burst. The one is sending data, which in > the context of y two examples might be a good or bad idea, and the other is > receiving it. From the receiver's perspective, the data either arrived or > it didn't; if it arrived, there is no real argument for not delivering it > to its application... > From naeem.khademi at gmail.com Mon Dec 16 06:05:56 2013 From: naeem.khademi at gmail.com (Naeem Khademi) Date: Mon, 16 Dec 2013 15:05:56 +0100 Subject: [e2e] [aqm] What is a good burst? -- AQM evaluation guidelines In-Reply-To: References: <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com> <201312152257.rBFMveZO022075@bagheera.jungle.bt.co.uk> <80CEFF86-8687-4266-B87C-8FB5B894D4DF@cisco.com> Message-ID: and to clarify more about what else was discussed, it seems to me some of us tend to correspond and relate the notion of "good queue" vs. "bad queue" used by KN+VJ ACM queue paper to my question on "good bursts". While they likely to be correlated (I have no argument on this now), the notion of "good burst" goes beyond the "good queue" defined in that paper. Based on their definition a good queue is a queue that minimizes the standing queue (or gets rid of it entirely) while allowing a certain amount of (sub-RTT? typical 100 ms) bursts while avoiding the link to get under-utilized. That notion (again, I have no argument on its correctness for now) is different from my question on "good bursts" which means that: once we manage to get rid of the standing queue, what types/sizes of bursts I should let the AQM X to protect/handle? Naeem On Mon, Dec 16, 2013 at 2:47 PM, Naeem Khademi wrote: > Bob, Fred and all > > I'll copy/paste the question here again: "what is a good burst (size) > that AQMs should allow?" and/or "how an AQM can have a notion of the right > burst size?" > > So, obviously, as Bob mentioned, I'm concerned about what AQMs should or > shouldn't do. The mission of dealing with packet bursts in addition to the > task of keeping the standing queue very low or minimal is part of an "AQM > evaluation criteria" I envision. While I do agree with all Fred's remarks, > I'm more concerned to have an answer for this, for where AQMs might get > deployed. > > An example: when designing my AQM X should I care about 64K TSO-generated > bursts to safely pass without dropping or not? Does the answer (whatever > it is) also apply to the burst sizes typical of multimedia traffic, etc.? > if the answer is "yes", should an AQM design be actively aware of what > application layer does in terms of sending bursty traffic or not? and to > what extent if yes? > > Regards, > Naeem > > On Mon, Dec 16, 2013 at 8:34 AM, Fred Baker (fred) wrote: > >> >> On Dec 15, 2013, at 2:57 PM, Bob Briscoe >> wrote: >> >> > Fred, >> > >> > Jonathan Morton, Michael Scharf & I took Naeem's question to mean "What >> should an AQM assume the size of a good burst is?" whereas I think you and >> David C-B took the question to mean "What should an end-system take the >> size of a good burst to be?". >> >> I can't comment on what he means. I took the question as "what should a >> system that is in receipt of what it might consider a 'burst', and more >> especially a 'good burst', to be?" >> >> I don't know that a sending transport (which is to be distinguished from >> the queueing arrangement in that same system) or a receiving system *has* a >> definition of a "good" or "bad" burst. The one is sending data, which in >> the context of y two examples might be a good or bad idea, and the other is >> receiving it. From the receiver's perspective, the data either arrived or >> it didn't; if it arrived, there is no real argument for not delivering it >> to its application... >> > > From fred at cisco.com Mon Dec 16 09:30:03 2013 From: fred at cisco.com (Fred Baker (fred)) Date: Mon, 16 Dec 2013 17:30:03 +0000 Subject: [e2e] [aqm] What is a good burst? -- AQM evaluation guidelines In-Reply-To: References: <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com> <201312152257.rBFMveZO022075@bagheera.jungle.bt.co.uk> <80CEFF86-8687-4266-B87C-8FB5B894D4DF@cisco.com> Message-ID: <84929DB4-4034-4B45-8BAA-D262076E9986@cisco.com> On Dec 16, 2013, at 6:05 AM, Naeem Khademi wrote: > and to clarify more about what else was discussed, it seems to me some of us tend to correspond and relate the notion of "good queue" vs. "bad queue" used by KN+VJ ACM queue paper to my question on "good bursts". While they likely to be correlated (I have no argument on this now), the notion of "good burst" goes beyond the "good queue" defined in that paper. Based on their definition a good queue is a queue that minimizes the standing queue (or gets rid of it entirely) while allowing a certain amount of (sub-RTT? typical 100 ms) bursts while avoiding the link to get under-utilized. That notion (again, I have no argument on its correctness for now) is different from my question on "good bursts" which means that: once we manage to get rid of the standing queue, what types/sizes of bursts I should let the AQM X to protect/handle? I think your question has a problem in it. Going back to my thought experiment, suppose that we have a queuing point whose egress speed is X and a sender that is sending data in a CBR fashion at (1 + epsilon)*X. In a very formal sense, the entire transmission stream is a single burst, and one could imagine it taking hundreds or thousands of packets being sent and forwarded before the queue built up to a point that AQM would push back. In that case, I would expect an "acceptable burst" to be hundreds or thousands of packets. If on the other hand you have a new TCP session in slow-start that is using an intermediate link that is at the time fully utilized and on the cusp of AQM pushing back on it, the new session is very likely to tip the balance, and a burst of a few packets might well push it over the top. So to my mind, the question isn't about the size of the burst. It is about the rate of onset and the effect of that burst on the latency and probably of loss for itself and competing sessions. And it will never come down to a magic number N in that N is somehow "right", N-1 is "better", and N+1 is "over the top." There are no such magic numbers. > Naeem > > > On Mon, Dec 16, 2013 at 2:47 PM, Naeem Khademi wrote: > Bob, Fred and all > > I'll copy/paste the question here again: "what is a good burst (size) that AQMs should allow?" and/or "how an AQM can have a notion of the right burst size?" > > So, obviously, as Bob mentioned, I'm concerned about what AQMs should or shouldn't do. The mission of dealing with packet bursts in addition to the task of keeping the standing queue very low or minimal is part of an "AQM evaluation criteria" I envision. While I do agree with all Fred's remarks, I'm more concerned to have an answer for this, for where AQMs might get deployed. > > An example: when designing my AQM X should I care about 64K TSO-generated bursts to safely pass without dropping or not? Does the answer (whatever it is) also apply to the burst sizes typical of multimedia traffic, etc.? if the answer is "yes", should an AQM design be actively aware of what application layer does in terms of sending bursty traffic or not? and to what extent if yes? > > Regards, > Naeem > > On Mon, Dec 16, 2013 at 8:34 AM, Fred Baker (fred) wrote: > > On Dec 15, 2013, at 2:57 PM, Bob Briscoe > wrote: > > > Fred, > > > > Jonathan Morton, Michael Scharf & I took Naeem's question to mean "What should an AQM assume the size of a good burst is?" whereas I think you and David C-B took the question to mean "What should an end-system take the size of a good burst to be?". > > I can't comment on what he means. I took the question as "what should a system that is in receipt of what it might consider a 'burst', and more especially a 'good burst', to be?" > > I don't know that a sending transport (which is to be distinguished from the queueing arrangement in that same system) or a receiving system *has* a definition of a "good" or "bad" burst. The one is sending data, which in the context of y two examples might be a good or bad idea, and the other is receiving it. From the receiver's perspective, the data either arrived or it didn't; if it arrived, there is no real argument for not delivering it to its application... > > Make things as simple as possible, but not simpler. Albert Einstein From detlef.bosau at web.de Mon Dec 16 13:37:33 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Mon, 16 Dec 2013 22:37:33 +0100 Subject: [e2e] [aqm] What is a good burst? -- AQM evaluation guidelines In-Reply-To: <84929DB4-4034-4B45-8BAA-D262076E9986@cisco.com> References: <9660C545-473C-4039-AB42-A12B7C761FC8@cisco.com> <201312152257.rBFMveZO022075@bagheera.jungle.bt.co.uk> <80CEFF86-8687-4266-B87C-8FB5B894D4DF@cisco.com> <84929DB4-4034-4B45-8BAA-D262076E9986@cisco.com> Message-ID: <52AF729D.1020604@web.de> Am 16.12.2013 18:30, schrieb Fred Baker (fred): > I don't know that a sending transport (which is to be distinguished from the queueing arrangement in that same system) or a receiving system *has* a definition of a "good" or "bad" burst. The one is sending data, which in the context of y two examples might be a good or bad idea, and the other is receiving it. From the receiver's perspective, the data either arrived or it didn't; if it arrived, there is no real argument for not delivering it to its application... > Particularly, CoDel is full of magic numbers. That's the reason of my doubts. -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From kojo at cs.helsinki.fi Mon Dec 16 15:03:59 2013 From: kojo at cs.helsinki.fi (Markku Kojo) Date: Tue, 17 Dec 2013 01:03:59 +0200 (EET) Subject: [e2e] Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> Message-ID: Hi Andrew, catching up with this thread ... On Fri, 6 Dec 2013, Andrew Mcgregor wrote: > I'm running a fairly standard OpenWRT QoS config, with ingress and egress > shaping to prevent (too much) upstream queueing either in the network or in > the modem; I get about 22Mbps symmetric performance on this particular > setup, with reasonable latencies (I wouldn't be happy if I was FPS gaming, > but it's fine for everything else). I don't remember the latency numbers > off the top of my head. It's actually one of the better broadband > connections available in Australia, it's just a shame about the cost. This is clever way of making things work better over LTE. If I understood your setup correctly, you have moved the bottleneck (partially?) away from the LTE link by using shaping? In my view this, however, does not provide an empirical proof that fq_codel works fine over a LTE bottleneck. It's the shaping (together with fq for uplink) that makes the trick. Please correct me if I am wrong. It is not only fq_codel that is likely to have problems over most wireless technologies but any proposed AQM today has hard time to work properly over wireless bottleneck links like LTE, 3G, GPRS, wifi, satellite, etc. For various reasons (some well justified some not so), these technologies tend to buffer significant amount of data below the IP layer (as Keith indicated), quite totally hidden from your favorite IP AQM/scheduling, i.e., without a very specific cross-layer implementation RED does not know the actual queue length, PIE is not able to correctly figure out the departure rate, and fq_codel is not able to calculate the correct sojourn time nor drop or schedule the packets from the actual heads of the queues. This is not to say that a properly working AQM/scheduling is not doable, but there is a big obstacle as long as we continue using such a strict layering model between the IP and link layer and enforce it in the implementations. Cheers, /Markku > On 6 December 2013 11:16, Keith Winstein wrote: > >> On Thu, Dec 5, 2013 at 6:22 PM, Andrew Mcgregor wrote: >> >>> By the way, fq_codel is not a combination of SFQ and CoDel, but something >>> else. The code deserves really close study to understand what is really >>> going on there, because there is a very interesting contribution in terms >>> of heuristically measuring whether flows are building queue in the qdisc >>> or >>> not. Empirically, this works really well on user access links, including >>> LTE, by my own direct observation having run it on my home network's LTE >>> link for many months now. >> >> >> Andrew, I'd love to hear more about how you do this. Do you run fq_codel >> on the uplink or somehow also on the downlink? How do you prevent the LTE >> modem or baseband from buffering hundreds of kilobytes inside itself -- do >> you use an upstream traffic shaper (to prevent the LTE link from becoming >> the bottleneck) or something more sophisticated? >> >> Cheers, >> Keith >> > > From detlef.bosau at web.de Mon Dec 16 16:07:07 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Tue, 17 Dec 2013 01:07:07 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> Message-ID: <52AF95AB.7010100@web.de> Am 17.12.2013 00:03, schrieb Markku Kojo: > It is not only fq_codel that is likely to have problems over most > wireless technologies but any proposed AQM today has hard time to > work properly over wireless bottleneck links like LTE, 3G, GPRS, wifi, > satellite, etc. Christmas time is coming :-) (I have "Holidays are coming" in mind, however, I must not do any advertising here ;-)) Over time, I feared I were the only one to doubt AQM mechanisms over wireless bottleneck links. > For various reasons (some well justified some not so), these > technologies tend to buffer significant amount of data below > the IP layer (as Keith indicated), The major reason is to utilize the highly volatile throughput. And it is of major importance not to empty necessary buffers here e.g. as a consequence of congestion handling. > quite totally hidden from your > favorite IP AQM/scheduling, !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! that's exactly what makes splitting appealing in this context: Necessary buffers and their contents are hidden from TCP congestion control and queue management! > i.e., without a very specific cross-layer > implementation RED does not know the actual queue length, PIE is not > able to correctly figure out the departure rate, and fq_codel is not > able to calculate the correct sojourn time So, you understand my thoughts on this one? > nor drop or schedule the packets from the actual heads of the queues. > This is not to say that > a properly working AQM/scheduling is not doable, but there is a big > obstacle as long as we continue using such a strict layering model > between the IP and link layer and enforce it in the implementations. Thank you very much for posting this comment. > >> On 6 December 2013 11:16, Keith Winstein wrote: >> >>> On Thu, Dec 5, 2013 at 6:22 PM, Andrew Mcgregor >>> wrote: >>> >>>> By the way, fq_codel is not a combination of SFQ and CoDel, but >>>> something >>>> else. The code deserves really close study to understand what is >>>> really >>>> going on there, because there is a very interesting contribution in >>>> terms >>>> of heuristically measuring whether flows are building queue in the >>>> qdisc >>>> or >>>> not. Empirically, this works really well on user access links, >>>> including >>>> LTE, by my own direct observation having run it on my home >>>> network's LTE >>>> link for many months now. >>> >>> >>> Andrew, I'd love to hear more about how you do this. Do you run >>> fq_codel >>> on the uplink or somehow also on the downlink? How do you prevent >>> the LTE >>> modem or baseband from buffering hundreds of kilobytes inside itself >>> -- do >>> you use an upstream traffic shaper (to prevent the LTE link from >>> becoming >>> the bottleneck) or something more sophisticated? >>> >>> Cheers, >>> Keith >>> >> >> > -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From richard at bennett.com Mon Dec 16 17:10:26 2013 From: richard at bennett.com (Richard Bennett) Date: Mon, 16 Dec 2013 17:10:26 -0800 Subject: [e2e] Codel and Wireless In-Reply-To: <52AF95AB.7010100@web.de> References: <52963EF4.7000908@web.de> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> <52AF95AB.7010100@web.de> Message-ID: <52AFA482.7060707@bennett.com> It seems to me that indifference to applications and link layer conditions is not really a productive. On 12/16/13, 4:07 PM, Detlef Bosau wrote: > Am 17.12.2013 00:03, schrieb Markku Kojo: >> It is not only fq_codel that is likely to have problems over most >> wireless technologies but any proposed AQM today has hard time to >> work properly over wireless bottleneck links like LTE, 3G, GPRS, wifi, >> satellite, etc. > > Christmas time is coming :-) (I have "Holidays are coming" in mind, > however, I must not do any advertising here ;-)) > > Over time, I feared I were the only one to doubt AQM mechanisms over > wireless bottleneck links. > > >> For various reasons (some well justified some not so), these >> technologies tend to buffer significant amount of data below >> the IP layer (as Keith indicated), > The major reason is to utilize the highly volatile throughput. > > And it is of major importance not to empty necessary buffers here e.g. > as a consequence of congestion handling. > >> quite totally hidden from your >> favorite IP AQM/scheduling, > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > > that's exactly what makes splitting appealing in this context: Necessary > buffers and their contents are hidden from TCP congestion control and > queue management! > >> i.e., without a very specific cross-layer >> implementation RED does not know the actual queue length, PIE is not >> able to correctly figure out the departure rate, and fq_codel is not >> able to calculate the correct sojourn time > > So, you understand my thoughts on this one? >> nor drop or schedule the packets from the actual heads of the queues. >> This is not to say that >> a properly working AQM/scheduling is not doable, but there is a big >> obstacle as long as we continue using such a strict layering model >> between the IP and link layer and enforce it in the implementations. > Thank you very much for posting this comment. >>> On 6 December 2013 11:16, Keith Winstein wrote: >>> >>>> On Thu, Dec 5, 2013 at 6:22 PM, Andrew Mcgregor >>>> wrote: >>>> >>>>> By the way, fq_codel is not a combination of SFQ and CoDel, but >>>>> something >>>>> else. The code deserves really close study to understand what is >>>>> really >>>>> going on there, because there is a very interesting contribution in >>>>> terms >>>>> of heuristically measuring whether flows are building queue in the >>>>> qdisc >>>>> or >>>>> not. Empirically, this works really well on user access links, >>>>> including >>>>> LTE, by my own direct observation having run it on my home >>>>> network's LTE >>>>> link for many months now. >>>> >>>> Andrew, I'd love to hear more about how you do this. Do you run >>>> fq_codel >>>> on the uplink or somehow also on the downlink? How do you prevent >>>> the LTE >>>> modem or baseband from buffering hundreds of kilobytes inside itself >>>> -- do >>>> you use an upstream traffic shaper (to prevent the LTE link from >>>> becoming >>>> the bottleneck) or something more sophisticated? >>>> >>>> Cheers, >>>> Keith >>>> >>> > -- Richard Bennett Visiting Fellow, American Enterprise Institute Center for Internet, Communications, and Technology Policy Editor, High Tech Forum From detlef.bosau at web.de Tue Dec 17 04:19:01 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Tue, 17 Dec 2013 13:19:01 +0100 Subject: [e2e] Codel and Wireless In-Reply-To: <52AFA482.7060707@bennett.com> References: <52963EF4.7000908@web.de> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> <52AF95AB.7010100@web.de> <52AFA482.7060707@bennett.com> Message-ID: <52B04135.2030203@web.de> Am 17.12.2013 02:10, schrieb Richard Bennett: > It seems to me that indifference to applications and link layer > conditions is not really a productive. > I'm totally with you! However: It's exactly that what TCP actually does! (Although "indifference" is a bit obfuscated by network- or path-models which pretend to be "generic", nevertheless they are actually wrong!) Detlef -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From detlef.bosau at web.de Thu Dec 19 03:04:25 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Thu, 19 Dec 2013 12:04:25 +0100 Subject: [e2e] What are the shared resources? Re: Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <20131127.202801.41673746.sthaug@nethelp.no> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> Message-ID: <52B2D2B9.7050402@web.de> In my opinion, the problem is located in a completely different place - and in my opinion, we should think about solving it eventually. The whole TCP congestion control issue, CoDel, AQM etc. are parts of which, is basically a matter of resource sharing. Does anyone contradict? O.k., so we can fix this. The relevant question is however not only, HOW this resources should be shared, the question is WHAT the shared resources are! Since Van Jacobsons paper from 1989, the shared resource is a "latency throughput product", sometimes even called (wrongly) "latency bandwidth product". From its physical dimension this product is a "capacity", regardless, whether there may be actually several packets on the line at the same time or not (WiFi, CSMA: Only one packet at a time is possible) or whether the "capacity" is used by packets and retransmitted data (mobile wireless networks), whathever the resources are, they are "made" into a "generic resource" "latency bandwidth product" - which is basically an artifact. Nevertheless, this artifact is shared. To my understanding, this abstraction had its justification in the late 80s, where the Internet was basically made up by a number of X.25 lines. However, I'm absolutely not convinced, that this model holds for the current structure of computer networks. Hence, our problem is that we share and manage a "resource" which actually does not exist but is an (IMHO inappropriate) "generic model" for what's going on in reality. Detlef Am 17.12.2013 00:03, schrieb Markku Kojo: > Hi Andrew, > > catching up with this thread ... > > On Fri, 6 Dec 2013, Andrew Mcgregor wrote: > >> I'm running a fairly standard OpenWRT QoS config, with ingress and >> egress >> shaping to prevent (too much) upstream queueing either in the network >> or in >> the modem; I get about 22Mbps symmetric performance on this particular >> setup, with reasonable latencies (I wouldn't be happy if I was FPS >> gaming, >> but it's fine for everything else). I don't remember the latency >> numbers >> off the top of my head. It's actually one of the better broadband >> connections available in Australia, it's just a shame about the cost. > > This is clever way of making things work better over LTE. > If I understood your setup correctly, you have moved the bottleneck > (partially?) away from the LTE link by using shaping? > In my view this, however, does not provide an empirical proof that > fq_codel works fine over a LTE bottleneck. It's the shaping > (together with fq for uplink) that makes the trick. Please correct > me if I am wrong. > > It is not only fq_codel that is likely to have problems over most > wireless technologies but any proposed AQM today has hard time to > work properly over wireless bottleneck links like LTE, 3G, GPRS, wifi, > satellite, etc. For various reasons (some well justified some not so), > these technologies tend to buffer significant amount of data below > the IP layer (as Keith indicated), quite totally hidden from your > favorite IP AQM/scheduling, i.e., without a very specific cross-layer > implementation RED does not know the actual queue length, PIE is not > able to correctly figure out the departure rate, and fq_codel is not > able to calculate the correct sojourn time nor drop or schedule the > packets from the actual heads of the queues. This is not to say that > a properly working AQM/scheduling is not doable, but there is a big > obstacle as long as we continue using such a strict layering model > between the IP and link layer and enforce it in the implementations. > > Cheers, > > /Markku > > >> On 6 December 2013 11:16, Keith Winstein wrote: >> >>> On Thu, Dec 5, 2013 at 6:22 PM, Andrew Mcgregor >>> wrote: >>> >>>> By the way, fq_codel is not a combination of SFQ and CoDel, but >>>> something >>>> else. The code deserves really close study to understand what is >>>> really >>>> going on there, because there is a very interesting contribution in >>>> terms >>>> of heuristically measuring whether flows are building queue in the >>>> qdisc >>>> or >>>> not. Empirically, this works really well on user access links, >>>> including >>>> LTE, by my own direct observation having run it on my home >>>> network's LTE >>>> link for many months now. >>> >>> >>> Andrew, I'd love to hear more about how you do this. Do you run >>> fq_codel >>> on the uplink or somehow also on the downlink? How do you prevent >>> the LTE >>> modem or baseband from buffering hundreds of kilobytes inside itself >>> -- do >>> you use an upstream traffic shaper (to prevent the LTE link from >>> becoming >>> the bottleneck) or something more sophisticated? >>> >>> Cheers, >>> Keith >>> >> >> > -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From detlef.bosau at web.de Sun Dec 22 04:58:56 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Sun, 22 Dec 2013 13:58:56 +0100 Subject: [e2e] What are the shared resources? Re: Codel and Wireless In-Reply-To: <52B2D2B9.7050402@web.de> References: <52963EF4.7000908@web.de> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> <52B2D2B9.7050402@web.de> Message-ID: <52B6E210.9010102@web.de> Admittedly, I expected major contradiction here? Or are my thoughts that weird that they don't deserve a comment? > -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From andrewmcgr at google.com Sun Dec 22 16:30:00 2013 From: andrewmcgr at google.com (Andrew Mcgregor) Date: Mon, 23 Dec 2013 11:30:00 +1100 Subject: [e2e] What are the shared resources? Re: Codel and Wireless In-Reply-To: <52B6E210.9010102@web.de> References: <52963EF4.7000908@web.de> <52966D03.7050909@web.de> <20131128.061701.74698037.sthaug@nethelp.no> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> <52B2D2B9.7050402@web.de> <52B6E210.9010102@web.de> Message-ID: Look up all the research on 'effective bandwidth' and 'grade of service' (not class of service), and read some large-deviation statistics. Then go see the model of TCP as a stochastic differential equation. It comes together eventually. On 22 December 2013 23:58, Detlef Bosau wrote: > Admittedly, I expected major contradiction here? > > Or are my thoughts that weird that they don't deserve a comment? > > > > > -- > ------------------------------------------------------------------ > Detlef Bosau > Galileistra?e 30 > 70565 Stuttgart Tel.: +49 711 5208031 > mobile: +49 172 6819937 > skype: detlef.bosau > ICQ: 566129673 > detlef.bosau at web.de http://www.detlef-bosau.de > > -- Andrew McGregor | SRE | andrewmcgr at google.com | +61 4 1071 2221 From detlef.bosau at web.de Mon Dec 23 05:08:29 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Mon, 23 Dec 2013 14:08:29 +0100 Subject: [e2e] What are the shared resources? Re: Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> <52B2D2B9.7050402@web.de> <52B6E210.9010102@web.de> Message-ID: <52B835CD.1060407@web.de> Am 23.12.2013 01:30, schrieb Andrew Mcgregor: > Look up all the research on 'effective bandwidth' and 'grade of > service' (not class of service), and read some large-deviation > statistics. Then go see the model of TCP as a stochastic differential > equation. It comes together eventually. > Did you even read, what I wrote? You argue with exactly those models which I blamed to be artefacts. In addition, you MUST NOT (by no means ever) make a conjecture for short term behaviour from long term statistics. (Actually, this is a major mistake which I meed EXTREMELY often by reading papers on this matter.) What differential equations or difference equations are concerned: These look always fine in papers. However. With respect to computer networks their use is highly questionable. Particularly for mobile networks they are - and after dealing with this subject for fourteen years now this is a statement I thought about very much - no way appropriate. Detlef -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From detlef.bosau at web.de Mon Dec 23 13:02:24 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Mon, 23 Dec 2013 22:02:24 +0100 Subject: [e2e] What are the shared resources? Re: Codel and Wireless In-Reply-To: References: <52963EF4.7000908@web.de> <529777C2.6030308@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> <52B2D2B9.7050402@web.de> <52B6E210.9010102@web.de> Message-ID: <52B8A4E0.8030609@web.de> Am 23.12.2013 01:30, schrieb Andrew Mcgregor: > Look up all the research on 'effective bandwidth' and 'grade of service' Frankly spoken, I was upset by your post. What is "effective bandwidth"? Some other average - from what you derive short term forecasts from long term observations. This is no way a resource. Perhaps, it was a bit courageous to put in question the "end to end principle" and VJCC that frankly. Perhaps, it was stupid. However, the more I think about it, VJCC appears to me like a very effective approach to work around congestion collapses and retransmission collapses. However, as soon as we leave a qualitative understanding of VJCC and try to develop a quantitative understanding, the whole thing appears to me as a justification ex post. As far as I see, VJCC was developed within a certain context: TCP as designed in RFC 793 does not care for resources. As a result, we had several congestion collapses, or retransmission collapses respectively. VJCC attempted to work around these collapses: When a TCP flow suffered from packet loss, this was taken as an indication for congestion and the TCP flow was slowed down. (Actually, nothing was slowed down, on the one hand the sending window was halved, the relation between window and rate is a non trivial one, on the other hand, the RTO was doubled. Both lowered the load offered to the network by the sender.) All the quantitative interpretations of VJCC, rationales with Little's law, "Bandwidth Delay Product". all the TCP models where, yes it sounds harsh, in a sense hand waving. A posteriori interpretations of an extremely useful work around - which however never attempted to do any resource management. Admittedly, I would appreciate talking about this issue with colleagues here in Germany because the issue is extremely touchy and I would appreciate the opportunity of talking about these issues in my mother tongue. Unfortunately, I don't know anybody who is interested in this kind of discussion. Detlef -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de From detlef.bosau at web.de Tue Dec 24 06:24:08 2013 From: detlef.bosau at web.de (Detlef Bosau) Date: Tue, 24 Dec 2013 15:24:08 +0100 Subject: [e2e] What are the shared resources? Re: Codel and Wireless In-Reply-To: <52B8A4E0.8030609@web.de> References: <52963EF4.7000908@web.de> <5298930D.7000908@web.de> <529B5C68.9060202@web.de> <529E2CFB.9050307@web.de> <1386105079.17863.YahooMailNeo@web141601.mail.bf1.yahoo.com> <529F6907.7070003@web.de> <52A0A09F.40703@web.de> <52B2D2B9.7050402@web.de> <52B6E210.9010102@web.de> <52B8A4E0.8030609@web.de> Message-ID: <52B99908.4050002@web.de> Am 23.12.2013 22:02, schrieb Detlef Bosau: > Am 23.12.2013 01:30, schrieb Andrew Mcgregor: >> Look up all the research on 'effective bandwidth' and 'grade of service' > Frankly spoken, I was upset by your post. > > What is "effective bandwidth"? Some other average - from what you derive > short term forecasts from long term observations. > Perhaps, I'm allowed to look at "effective bandwidth" a bit closer. Apparently, this term characterizes an "achievable throughtput" in contrast to a "maximum throughtput". It makes sense to determine an achievable throughput in contrast to a maximum throughput because the maximum throughput is usually shared between several flows, in addition, retransmissions and protocol overhead must be taken into account. Neither of the aforementioned is assessed e.g. by packet pair or packet train approaches. Characterizing a link by an "achievable throughtput" is not new, I did so myself years ago: @inproceedings{pte, booktitle = "KiVS Kurzbeitr?ge und Workshop 2005", year = 2005, title = "{Path Tail Emulation: An Approach to Enable End--to--End Congestion Control for Split Connections and Performance Enhancing Proxies}", author = "D.~Bosau", address = "Kaiserslautern, Germany", pages = "33-40" } After my talk I was asked the simple question (and I'm grateful for this question): "How do you know the achievable throughput about a wireless link?" I uttered quite some nonsense about continuous monitoring and the like - and eventually had to understand that this question actually debunked my whole idea as nonsense. Actually, and I'm grateful for any correction here, it is actually impossible to asses somethink like an "effective bandwidth" or "achievable throughput" for a mobile wireless link. And although this is annoying: A resource distribution/control/management scheme which does not work for a single link hardly works for a path consisting of a sequence of links. The idea of hiding links or more complex parts of an Internet path behind some mechanism which emulates the part behind a "black box" with an "effective bandwidth" or an "effective latency" is appealing. We could distribute resources between flows, we wouldn't have any loss differentiation problems, no spurious timeouts, we had a "Kodachrome" network (credits to Simon & Garfunkel). IIRC, at the same conference Matthias Scheidegger presented an approach which simulated larger parts of the Internet with some kind of emulation model for the NS2. As I said. The idea is appealing. However, it doesn't work for all links. Scheidegger's approach however is useful for simulation purposes. One can simulate an observed throughput. So you can simulate what has already happened. My fault was that I tried to emulate an "effective throughput" for an operational mobile wireless link. I would greatly appreciate someone who tells me, that I'm wrong. I would be eager to proceed the work with PTE - however, I didn't to so because I had to understand that my approach was wrong. -- ------------------------------------------------------------------ Detlef Bosau Galileistra?e 30 70565 Stuttgart Tel.: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 detlef.bosau at web.de http://www.detlef-bosau.de