Disappointing bufferbloat

[old-hacker]

I recently switched from DD-WRT to Gargoyle (1.8.0) on my TP-Link WRT4300 router. I've been testing basic QoS on both using just the basic default values, with no added rules using the speedtest at dslreports.com, which has a score for bufferbloat.

With DD-WRT, I was often getting a score of A or A+, but I really didn't like the firmware, so I decided to try Gargoyle. But with all my tweaking of speeds, I can't get a bufferbloat score higher than D. Even cutting my upload speed down to almost half, using ACC for download, and cutting download speed to about 80% didn't help.

I ran a netalyzer test, and it didn't show any problems at all, which adds to my confusion about what is going on. I thought I was getting a handle on controlling bufferbloat, but I'm beginning to think I don't know anything now.

I use VOIP for my home phone service, along with Netflix and other streaming, so that's the main reasons I want to use QoS. My home network is only used by my wife and I, and my son.

Is there anything else I need to do or know about to improve things?

[tapper]

The Best Tests for bufferbloat:
http://www.bufferbloat.net/projects/cer ... ufferbloat

[old-hacker]

The Best Tests for bufferbloat:
http://www.bufferbloat.net/projects/cer ... ufferbloat

Thanks. I had read that page, along with most everything else I could find. The tests I was doing was from the first link of that page.

But this time, I went further, and tried the other link, thinkbroadband.com, and also I got hold of 2 of their scripts (betterspeedtest.sh, and netperfrunner.sh). Assuming I am interpreting all the results correctly, they all show good performance. The two scripts at first showed high latency, using all default values, but then I read about specifying the server, and there is one in California (my location), so I redid them and they came out good. So maybe there is something about the dslreports test and Gargoyle that are giving inaccurate results.

I did try pinging Google continuously while doing a normal speedtest, and the ping time increased to a maximum 450 ms (normal 15 ms) during the upload test. That didn't make me too happy.

I guess I will need to dig deeper.

[orangetek]

Do the following, and test again. Make sure you change your total upload/download bandwidth accordingly.

Step 1.
https://www.dropbox.com/s/gsnyhwzk5f2s8 ... 2.JPG?dl=0

Step 2.
https://www.dropbox.com/s/v6lo03f105dxn ... 1.JPG?dl=0

Step 3.
https://www.dropbox.com/s/hm0k9hsgi98ba ... 2.JPG?dl=0

Step 4.
https://www.dropbox.com/s/cgiku87hzuhuf ... 1.JPG?dl=0

[old-hacker]

Do the following, and test again. Make sure you change your total upload/download bandwidth accordingly.

Step 1.
https://www.dropbox.com/s/gsnyhwzk5f2s8 ... 2.JPG?dl=0

Step 2.
https://www.dropbox.com/s/v6lo03f105dxn ... 1.JPG?dl=0

Step 3.
https://www.dropbox.com/s/hm0k9hsgi98ba ... 2.JPG?dl=0

Step 4.
https://www.dropbox.com/s/cgiku87hzuhuf ... 1.JPG?dl=0

That improved the score considerably - from D to A (quality remained A). I'll have to play around with the numbers to see what makes the difference - TBH, so far, I don't really see how it works. The percentages for fast and slow are different, but not a lot different, than what I had, but I don't understand about adding a new classification rule for fast affects a normal classified test. But I don't really know what it does at all, so there could be a reason. Do you have any explanation for that change?

[orangetek]

It works because anything other than important packets (0-512 bytes) go to the default service class. You could set the percentages 50/50 and it would work the same because the packets in the classification rule would never use 50% of your bandwidth. Understanding what those packets are and why they need to be prioritized first is another story. Google TCP ACK and go from there. Just remember, whatever classification rules you create, the default service class catches everything that doesn't match and should always be set to your slowest service class.

I run a wisp in Cyprus with 400+ customers connected. Without proper qos i would not have a business.

[pbix]

Very interesting orangetek. What router do you use for your Cyprus WISP and what is your interest in Gargoyle?

[orangetek]

Hi pbix, we have 2 120mb/8mb wan connections, each connected to an x86 gargoyle box modded for quad core/4gb. They both go to a 9 core MikroTik cloudCore router that handles the load balancing for both wans and also runs a customized hotspot for our clients. Our interest in Gargoyle is the ACC because our wan speed drop slightly during peak hours BUT ACC for us is not working well due to our provider randomly dropping pings. Pings on average to 8.8.8.8 are 15ms. When a timeout comes along, there is no change in ms before or after the timeout so it's not a bandwidth issue, probably my provider filtering icmp above a certain limit at their end. I think i suggested somewhere on the forum to have an option to ignore timeouts. Gargoyles qos is also very good compared to others even if lacks a few options, it works very well with minimum hassle. I am currently looking at the source code for ACC to see if i can mod it to ignore ping timeouts but i don't really know much C. I noticed that if i ping the same target from 2 different consoles, they don't timeout in sync. In fact, one console may timeout once in 100 pings and the other 10 times out of 100. It's random and not bandwidth or connection related.
I was thinking along the lines of, when a timeout occurs, replace the value with the previous value before the timeout. Should be easy to implement?

[pbix]

If you can build your own version of Gargoyle here is something you can try.

The ACC function is contained in the file qosmon.c. Locate this file and search for the following lines of code:

Code: Select all

   

    while (!sigterm) {
        int len = sizeof (packet);
        socklen_t fromlen = sizeof (from);
        int cc;
        u_char chill;
        struct timeval timeout;
        fd_set fdmask;
        FD_ZERO(&fdmask);
        FD_SET(s , &fdmask);

        //This variable will be set in pr_pack() if we get a pong that matched
        //our ping, but in case we don't get we initialize to the period.
        rawfltime=period;

and change to this.

Code: Select all

 
   //Initialize plimit to a sane value.
    plimit = period*1000;
	
    while (!sigterm) {
        int len = sizeof (packet);
        socklen_t fromlen = sizeof (from);
        int cc;
        u_char chill;
        struct timeval timeout;
        fd_set fdmask;
        FD_ZERO(&fdmask);
        FD_SET(s , &fdmask);

        //This variable will be set in pr_pack() if we get a pong that matched
        //our ping, but in case we don't get we initialize to the current plimit * 2
        rawfltime=plimit/500;

Dropped ICMP packets are strong indication of a saturated link and it does not seem correct to me to do nothing in response. The new code will more gradually drop the link limit in response to dropped packets.

Please report on your results in a new thread.

[orangetek]

Thanks pbix, ill try it this evening during peak hours when the problem starts. I am going on the assumption that a ping timeout cause by bandwidth starvation will show a rapid increase in RTT before the timeout occurs. If ACC is fast enough at reducing the max limit then a timeout should not even occur. So if one does occur then its more than likely another reason why it got dropped. Just thinking out aloud