New qos performing worse

[hotzenpl0tz]

Hi, I am having some problems configuring the new qos system do work as good for me as the old one did. One of my main usages was to limit download and upload transfers to certain ports to a slow class, so that if I watch youtube or other stremaing content, it gets immediately throttled to very low speeds and I can watch the streams in rela-time. With the new system I am having trouble getting this to work, maybe my settings aren't correct.

My relevant classes are:
Fastest 40% 120 kbit/s unlimited
Slow 10% zero unlimited

My expectations are: when there is no usage from the faster classes, the slow class can use all the bandwidth. If there is bandwidth used by the faster classes, it should throttle down to very slow speeds - 800kbit would be perfectly fine (I have about 5800kbit downstream). This used to work before with older gargoyle versions, although I think I used a custom layer7 protocol for detecting http-video streams and gave them the min-delay class. Now I set the relevant ports of my download app to the slow class and the streaming is in the fastest class - this works, when I watch the connections in the gargoyle webinterface, they are correctly classified as fastest and slow in both directions. But my download, although being in the slow class is still at about 500kbyte/s and my youtube stream - a 720p file that needs quite a bit of bandwidth to be watched without hickups - is downloading at maybe 100kbyte/s.

Any pointers ?

[pbix]

So I think I understand your setup.

You have two classes at 40% and 10%. Now this does not add up to 100% so do you have other classes as well?

It is incorrect to say that as soon as the fastest class starts the slow class will be throttled to 10%. The slow class will be throttled only if the link reaches capacity. This is why the column title says "Bandwidth at capacity". Let say you had 'Fastest' at 90% and 'Slow" at 10%. If the streaming requires only 80% of the link then the 'Slow' class would get the remaining 20%. The only time you will see the slow class at 10% is if the 'Fastest' class needs its full share of 90% of the link.

You say your video is choppy but do we know that it is choppy because of the lack of bandwidth? For example if the server that was sending the file was slow in sending then you would see the issue you state. How much bandwidth is needed to watch your video without choppiness?

[hotzenpl0tz]

Hi pbix - thanks for helping me out. I have gone through my settings again, reudjusted them a bit and have gotten better results now since I had a wrong value for my total bandwidth when I first posted this, although not quite where I want it to be yet - but maybe you can give me some pointers on the right way to go:

The following shows a test I ran. The blue graph represents my fastest class, the red one my slow class. I started a youtube 720p video at the beginning and let it run for a while. The download speed was just slightly faster then the speed I needed to watch the video without hiccups, so the difference between the point in the video and the point the movie has downloaded to in the background was slightly increasing over time.
When the slow class beings in the graph, I started my download - this download maxes my line speed completely as soon as I start it.
What happens is, that at first the download from the youtube stream in the background gets alot slower until the qos implementation adjusts (this is to be expected of course) and then my download does get scaled down - but I think the problem is that it isn't scaled done quite enough - as you can see the youtube download bandwidth isn't quite steady but a bit dynamic and I think that is why it isn't working properly as a sudden spike in bandwidth causes the problem.


My setup:



What I am trying to achieve is to get downloads to take all the bandwidth available if only they are running, but as soon as there a high bandwidth demands for example from youtube streams or other html requests, I need the download to be scaled down, I don't care at all if it gets scaled down more than is needed in order to give my video some scaling space. Is there a way to do that ? (I thought about giving the fastest class a guaranteed speed of 80% of my total bandwidth, but I am not sure that is a good thing?).

[pbix]

It seems to me that you are saying you want all your classes to have priority over your slow class. If that is the case then just lower you slow class to 1%. Then when there is competition in the link it will always lose.

I cannot tell the meaning of the colors in your bandwidth graph. Next time post a screen shot of the whole screen so we can see what you are trending.

You have 1024kbps as the minimum for one of your classes. Is this your video class? Did you test to see if 1024kbps would sustain your video throughput?

Finally, make sure to carefully inspect your connection list to make sure your video connections are being put it the fast class.

Finally how much downstream bandwidth do you actually have and how did you determine it?

[hotzenpl0tz]

Hm, though my explanation of the colors would suffice, but I agree that I maybe had too much text
Blue: fast class (video)
Red: slow class (download)

My complete bandwidth is 5100kbit - I determined this through several online speedtests and the fact that I can pretty much max out my line with the download I have in my slow class and that computes to abit more than 5100 - so it should be pretty acurate, maybe slightly lower than what I actually have, but not more.

As one can see on the graph, max download speed for that 720p video seems to be a bit more than 230kbyte/sec - so probably about 1700kbit/s - it seems youtube calculates how much speed is needed and sends with a slightly higher speed, but never maxes my line. So that answers one of your questions, the minimum I set for the class wouldn't suffice alone.
My understanding (and maybe misunderstanding) was, that if I set my fast class to 40% and my slow class to 10% the total bandwidth would be distributed something like this (when only those 2 classes are active):

Total bandwidth: 5100kbit
Fast class: 1024kbit (min bw) + 0.75*(5100-1024kbit) = 4081kbit
Slow class: 0.25*(5100-1024kbit) = 1019kbit
Since the ratio of fast to slow would be 4:1 (40% to 10%).

Now since the youtube stream only needs about 1700kbits, the rest that is freed up is probably given to the download class if I understand that right.

Now I don't actually know exactly how things work obviously The 4081kbit would be more than enough to satisfy the link (I need only about 1700kbits), but for some reason when both stream and download are active, the stream does not work without hiccups - i.e. I don't seem to get the same speed from youtube anymore.

[hotzenpl0tz]

Ah, your suggestion of trying to put my slow class to 1% seems to have helped here, it now barely has enough speed to keep the stream alive. Interestingly still lower streaming download speeds than with no download in the background, but this seems ok for my purposes.

[pbix]

For the record your fast connection will get 1024kbit/sec or its link share whichever is greater (ie they do not add). In the scenario you gave its link share would be 5100 * 40/(40+10) = 4080kbit/sec. Well that is the way it supposed to work anyway. If you set its minimum bandwidth at 1700kbit/sec then you should not need to worry about the share as it would have enough regardless of its share. Of course it only gets its link share if it actually uses it so in your case only 1700kbit/sec does not use the entire share and the rest as you say would go to the class that needs it.

I have been experimenting with improving the QoS some. If you are interested in trying my newest script you could copy the attached to your etc/init.d directory replacing the one that is there. If you try it please post your impressions.

[hotzenpl0tz]

Thanks for clarifying that, always good to know how things work exactly when playing with the parameters, makes it less of a guessing game

I have now downloaded and installed your new qos script, and one thing I noticed right away is that now my youtube stream seems to work alot better. I still have my slow class set to 1%, but now the youtube download seems to max out in the background and the download gets severly limited, exactly what I wanted. This might of course also be due to the dynamics of the internet that always make testing so difficult since the parameters change so often, but still.

Anything particular you changed that I can look out for ?

[pbix]

This script uses a more advanced queue. From the command line you can run "qos_gargoyle show" to see the entire QoS setup if you like. The new script uses RED queues. These queues require several parameters to set them up and getting the right parameters seems not to be very straight forward. I have been experimenting on my own box and noticed some improvement in smoothness of operation here.

Before I release something I like to get some other feedback from someone like yourself to make sure I did not screw something up.

What to watch? Well that the performance is closer to the target performance specified by your rules. Also that the class flows are smooth and we don't run out of memory. Rather subjective stuff to begin with, then when you throw in the fickleness of the internet in general its difficult to be sure which is better.

Based on your results and mine I guess I will update the SVN so that this script is included next time Eric does a Gargoyle release.