[e2e] query reg improving TCP performane

query query.cdac at gmail.com
Tue Jul 10 05:04:44 PDT 2007


Hi All,

    I was able to identify the size of the buffer in the Router's interface
. It was found to be 2040
    bytes and the buffer management scheme used is FIFO .
    Then based on this  statement by  Lachlan , I did the following tunings.

   ""
       The general rule-of-thumb for Reno is that the send buffer should be
       at least twice the bandwidth*RTT.  For BIC is is probably reduced to
       about 120% of the BDP (because it reduces its window by a smaller
       factor when there is a loss).  The receive buffer should still be at
       least equal to the BDP plus the router buffer.
                                                                       ""
   The tunings are as follows.

                 Send buffer = BDP + 120 % of  BDP
                                   =  921600  + 184320
                                   = 1105920 bytes

                Receive buffer =  BDP + Router's buffer size
                                      =  921600 +  2040
                                      =  923640 bytes

     After that I tried the same earlier experiments with iperf . I got a
average throughput of  66
     Mbits/s .

     Next ,  I tried some more tuning .  I increased the the send  buffer
size to twice of BDP.
     The receive buffer is same as earlier. This is according to what
Lachlan suggested for TCP
     Reno.
     Based on these tunings , I got a throughput of average 78 Mbits/s
using iperf. More improvement
     but not fully utilise the link capacity.

     But if I tune the window size to twice the size of BDP , I got  a
average throughput of
     around 88 Mbits/sec which I feel very much O.K for a 100 MBits/sec link
. But it can be further
    improved. Also , earlier, I have written in this mailing list that when
I tuned the window size to twice
     of BDP , I was getting a throughput of 95 Mbits /s . That I was
referring to the maximum
    throughput .
    For all these experiments, I was using BIC.

     I also tried with the following tunings as suggested . But I didnot get
any difference.

                        /sbin/ifconfig eth0 txqueuelen 1000
                        /sbin/sysctl -w net.ipv4.tcp_adv_win_scale=3


    So , based on the result of all these experiments , I reach the
following conclusion.

   The receive buffer should  be at least equal to the BDP plus the router
buffer .
   The send buffer should be 20% more than BDP if you are using BIC.
   These tunings  will  probably try to utilise the maximum link capacity
provided
   the buffer size in  the intermediate router is equal to the BDP.
   This I cannot prove practically because it is not possible fot me to
increase the buffer
    size  to the  size  of BDP , because the network is not under my
administration.

   But since in most cases the network is under the control of ISP , so it
might not
   be possible for end users to know the size of the router's interface.
   In that case , the end to end send and receive buffer size should be
atleast
   equal to twice the size of BDP to obtain maximum throughput.  This
statement
   was reflected by my findings.

     It will be helpfull to me if everybody give there views on my
conclusion. If you
     have any contradiction , please write.

    With Thanks and Regards
    zaman



















On 7/6/07, query <query.cdac at gmail.com> wrote:
>
>
> Thanks a lot Andrew . It helped me to understand and I feel that my
> Tunings are O.K.
>
> >   I was doing some Bandwidth measurement test on a 100 mbs link with a
> > RTT
> > > of about 70ms.
> > >   Based on that, I calculated the BDP as follows.
> > >
> > >           BDP = Bandwidth * RTT
> > >               = 921600 bytes
> > >    I did the following adjustments. I increased the above calculated
> > BDP by
> > > nearly
> > >    half of the value . The TCP settings now look like this.
> > >
> > >            /proc/sys/net/core/rmem_max  175636
> > >            /proc/sys/net/core/wmem_max  175636
> > >            /proc/sys/net/ipv4/tcp_rmem      4096    87380
> > > 175636
> > >            /proc/sys/net/ipv4/tcp_wmem      4096    87380
> > > 175636
> > >
> > >     After these settings , I find the link utilisation to be nearly 95
> > mbs.
> > >
> > >     According to many papers that I read , I found that the BDP should
> > be
> > > equal
> > >     to the product of Bandwidth * RTT .
> >
> > The papers probably said that *router* buffers need to equal the
> > bandwidth*RTT.  You are adjusting the sender/receiver buffers.  These
> > need to be significantly larger, as you have found.
>
>
>                       The papers or rather articles are talking of sender
> and receiver
>    buffers . Here is one such link where I find it.
>    http://www.psc.edu/networking/projects/tcptune/   .
>
>
>
> In order to allow retransmissions, the sender buffer needs to be able
> > to store all "packets in flight", which include both those in the in
> > the router buffers and those "on the wire" (that is, in the nominal
> > RTT of the link).
> >
> > In order to be able to provide in-order delivery, the receiver buffer
> > needs to be able to hold even more packets.  If a packet is lost, it
> > will receive an entire RTT (plus router buffer) worth of data before
> > the first retransmission of that packet will arrive.  If the first
> > retransmission is also lost, then it will need to store yet another
> > RTT worth of data.
> >
> > The general rule-of-thumb for Reno is that the send buffer should be
> > at least twice the bandwidth*RTT.  For BIC is is probably reduced to
> > about 120% of the BDP (because it reduces its window by a smaller
> > factor when there is a loss).  The receive buffer should still be at
> > least equal to the BDP plus the router buffer.
>
>
>    What I understand from your reply,  is that  It is not necessary that
> TCP Window should be
>    equal to BDP in all cases . Had the router buffer size is equal to  BDP
> ,  then I think I
>    should  equal link utilisation equal to the capacity of the link .
>    Since , In Internet it will not be  possible to know the router buffer
> size , so the best thing one
>    can  do , is to make the TCP window size twice to BDP as you have
> suggested.
>
>   I am finding another problem.  The UDP transmission rate on that link is
> decreased.  I changed
>   to the default settings , but it is showing the exact readings after
> tuning. It  seems   it  is  reading  some fixed  value from something  and
> based  on  that  it  is  transferring  data .
>
> The readings are like this..........
>
> iperf  -u -c 192.168.60.62 -t 300 -l 1460  -i 2
> ------------------------------------------------------------
> Client connecting to 192.168.60.62, UDP port 5001
> Sending 1460 byte datagrams
> UDP buffer size:  108 KByte (default)
> ------------------------------------------------------------
> [  3] local 10.128.0.2 port 32785 connected with 192.168.60.62 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3] -0.0- 2.0 sec   257 KBytes  1.05 Mbits/sec
> [  3]  2.0- 4.0 sec   257 KBytes  1.05 Mbits/sec
> [  3]  4.0- 6.0 sec   255 KBytes  1.05 Mbits/sec
> [  3]  6.0- 8.0 sec   257 KBytes  1.05 Mbits/sec
> [  3]  8.0-10.0 sec   255 KBytes  1.05 Mbits/sec
> [  3] 10.0-12.0 sec   257 KBytes  1.05 Mbits/sec
> [  3] 12.0-14.0 sec   255 KBytes  1.05 Mbits/sec
> [  3] 14.0-16.0 sec   257 KBytes  1.05 Mbits/sec
> [  3] 16.0-18.0 sec   257 KBytes  1.05 Mbits/sec
>
>    The result is for the following tuning.
>    net.core.rmem_default = 110592
>    net.core.wmem_default = 110592
>
>  After that I changed the tuning to
>             net.core.rmem_default = 196608
>             net.core.wmem_default = 196608
>
>    The readings for the tuning is like this...
> iperf  -u -c 192.168.60.62 -t 300 -l 1460  -i 2
> ------------------------------------------------------------
> Client connecting to 192.168.60.62, UDP port 5001
> Sending 1460 byte datagrams
> UDP buffer size:  192 KByte (default)
> ------------------------------------------------------------
> [  3] local 10.128.0.2 port 32785 connected with 192.168.60.62 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3] -0.0- 2.0 sec   257 KBytes  1.05 Mbits/sec
> [  3]  2.0- 4.0 sec   257 KBytes  1.05 Mbits/sec
> [  3]  4.0- 6.0 sec   255 KBytes  1.05 Mbits/sec
> [  3]  6.0- 8.0 sec   257 KBytes  1.05 Mbits/sec
> [  3]  8.0-10.0 sec   255 KBytes  1.05 Mbits/sec
> [  3] 10.0-12.0 sec   257 KBytes  1.05 Mbits/sec
> [  3] 12.0-14.0 sec   255 KBytes  1.05 Mbits/sec
>
>  Kindly please help me to rectify the problem. It is the same link on
> which I
>  performed the TCp test.
>
>   regards
>   zaman
>
> >
> >
>
>
>
>
>
>
> I hope this help,
> > Lachlan
> >
> > --
> > Lachlan Andrew  Dept of Computer Science, Caltech
> > 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
> > Phone: +1 (626) 395-8820    Fax: +1 (626) 568-3603
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20070710/20e8835c/attachment-0001.html


More information about the end2end-interest mailing list