Page 1 of 1

Question on QoS capabilities

Posted: Tue Nov 05, 2013 1:40 pm
by SirMaster
I read through the QoS wiki page but I have a few more questions of what's possible to do in Gargoyle.

I am currently using an old 2011 DDWRT build on my WNDR3700 and am looking to get a more featured QoS engine so i am looking at getting Gargoyle.

Here is what I want to be able to achieve. A multi-tiered bandwidth allocation setup.

Note that this is all just for upload.

At the bottom level I want to have my CrashPlan backups and torrents. I want these to be able to use all available bandwidth when nothing else is using any. They can fight over 50/50 of the available bandwidth.

I can classify this data by destination IP since crashplan goes out to a specific IP. I can classify the torrent data as data that comes out of a specific port (4444) on my server.

Next I want a level that is for users downloading files via FTP from my server. I can classify this as data going out over port 21.

I want this to also be able to use 100% of the bandwidth and make the lower crashplan/torrent tier go down to 1% when someone is downloading FTP from me.

Then I want a higher level yet for streaming video out to users. I want this to also be able to use 100% of the bandwidth, but to lower both the FTP and CrashPlan tiers to 1% (each?) when someone is trying to stream a video from my server.

I can classify this data again by outgoing port number.

Then finally I want a highest level tier which is basically "everything else" I want this tier to also be able to use up 100% of the bandwidth and lower down the streaming, FTP, and crashplan tiers to 1%.

This this setup possible in Gargoyle? Can I mix outgoing port classificaiton with destination IP classification? I can't use port classification for crashplan because it uses port 443 and I dont want to cripple HTTPS.

Also is it possible to make everything else that's not classified inherintly higher priority than all the stuff i listed to be classified?

Right now I use the QoS in DD-WRT and basically have crashplan, torrents, ftp, and streaming all set to "Bulk" priority. This works as in when nobody is using any bandwidth, crashplan takes it ALL. And even when I go to play a video game like DotA 2, I see that crashplan still is taking about 95% of my upload bandwidth yet my ping is not affected.

This is excellent and is very important. What I like is that i didnt have to classify the DotA 2 data. In fact, all my games that I play all work great and never lag even though Crashplan is uploading at 4.9mbps on my 5mbps line. I would like to have a similar setup where I can avoid classifying every single game I play (all unclassified traffic is above all classified traffic) since this changes all the time and stuff.

The real reason for improvement I am looking for is that FTP downloads are often very slow because they are fighting equally with CrashPlan. Also, my streaming does not work very well at all since it also has to fight with CrashPlan and sometimes FTP and torrents.

What do you guys think? Can I get my plan set up and working in Gargoyle through the GUI?

Thanks!

Re: Question on QoS capabilities

Posted: Tue Nov 05, 2013 4:45 pm
by Wisse
You can more or less achieve everything with Gargoyle's QoS settings via GUI.

I'm not sure if you can completely throttle certain services, but you can come close with smart settings. You should probably read this Wiki page, but I'll give you a very quick rundown anyway:

You already know how to classify your traffic, which is more or less the hardest part. Now you just have to create classes.
Basic variable for each class is "Percent Bandwidth", which you can obviously use to divide BW among classes. This is a description from Gargoyle's GUI:
Percent bandwidth at capacity is the percentage of the total available bandwidth that should be allocated to this class when all available bandwidth is being used. If unused bandwidth is available, more can (and will) be allocated. The percentages can be configured to equal more (or less) than 100, but when the settings are applied the percentages will be adjusted proportionally so that they add to 100.
DD-WRT most likely uses same thing, hidden behind user friendly names (eg. bulk).

So with this you could for example set 1% to CrashPlan/Torrents, 3% to FTP, 20% to video and 76% to everything else (you can set default class).
The downside of this is, as I said at the beginning, that you can't completely throttle down classes. Say you use example classes above and run Torrent and FTP at the same time, then your 5Mbit BW would be split to 1,25/3,75. If you run Torrent, FTP and Video it would be 0,2/0,6/4,16...
You can play around with percentages to get different ratios.

Some interesting functionality can also be gained with smart rules like the ones based on "traffic bytes" - few are already set in default QoS when you flash.

You can use "Min BW" for Dota 2 and other games. Gargoyle's description:
Minimum bandwidth specifies the minimum service this class will be allocated when the link is at capacity. For certain applications like VoIP or online gaming it is better to specify a minimum service in bps rather than a percentage. QoS will satisfiy the minimum service of all classes first before allocating the remaining service to other waiting classes.
Check wiki and this thread, which conveniently also talks about Dota 2.

I don't know DD-WRT that well, but I would say that on QoS front you can't go wrong with migration to Gargoyle since this is imo one of it's biggest strengths.

Re: Question on QoS capabilities

Posted: Tue Nov 05, 2013 5:29 pm
by SirMaster
Thanks for your descriptions, they did help. I did read that QoS page before I even posted, but it just wasn't exactly clear to me the possibilities.

I don't think that what I ideally want is easily possible from gargoyle though still.

Lets look at DD-WRT to see how it works now and what I am using so we can more easily see what I am trying to improve on.

Here is what DD-WRT says for how it works. This is basically all we need to know.

Priorities explained:
Exempt - Exempt no longer ignores global limits, instead it is now treated as "absolute max priority" higher than that of premium & is now the top bandwidth class guaranteeing 60% at minimum.
Premium - By default handshaking and icmp packets fall into this class. This class should be used sparingly. Occasionally VoIP and gaming services may be placed in this class so that they high priority.
Express - The Express class is for interactive applications (IRC, SSH, telnet, etc.) that require bandwidth above standard services so that interactive apps run smoothly.
Standard - All traffic that is not specifically classed will fall under the standard class. You should not need to explicitly set anything to this class.
Bulk - The bulk class is only allocated bandwidth when the remaining classes are idle. Use this class for P2P services and downloading services like FTP.

Detailed breakdown of traffic:
Bandwidth is allocated based on the following "minimum - maximum" percentages of uplink and downlink values for each class as of r21061 ~ current:
Exempt: 60% - 100%
Premium: 25% - 100%
Express: 10% - 100%
Standard: 5% - 100%
Bulk: 1% - 100%

What this really means is that if you have 10,000kbit of uplink traffic, "Standard" class traffic can be reduced and de-prioritized to 5% or 500kbit when a concurrent express or higher priority service requires the down/uplink pipe at the same time.

So right now, all my data is being put into Standard by default. Then I have CrashPlan, Torrents, FTP, and outgoing video streaming all set to bulk.

Now what I like about my current setup.

What I like is that it allows me to run as many uploading programs as I want at the same time and not have to worry about figuring out a bandwidth cap for any of them and it never affects any of my normal day-to-day traffic. I never had to tag dota, or battlefield, or sype, or remote desktop or any traffic. There are so many types of "normal" traffic. I do not want to have to identify and tag it all. This would take forever and would also change far too often. I have rooomates etc and I'm not going to be able to keep up with what new game they are playing and add new rules to tag their new game traffic as a high priority. I don;t think I shoul dneed to do that because IU dont have to now and everything day-to-day works fine.

Also what is nice is that my bulk traffic can use 100% of my if nobody is doing anytihng else (at night, or when everyone is at work).

Put simply, ALL my "normal" traffic is automatically prioritized above ALL the traffic that I tag as bulk upload traffic.

Now onto what I do not like about my current setup.

What I don't like is that Crashplan has to fight with FTP and has to fight with video streaming. Long story short, my parents for example stream video from my server and this takes 4Mbit when they do so. Currently If crashplan is backing up it and vidoe streaming are both fighting with the 5mbit and each gets about 2.5mbit. This means the video streaming does not work and constantly buffers until i pause crashplan.

Is there no way to say, when no video streaming is happening, crashplan can get 100% of my upload, but when video streaming starts, crashplan only gets what's left? In this case it would get 1Mbit because streaming would take 4Mbit.

But then on top of that, when someone starts to play video games, first crashplan should give up even more of it's speed to the video game because it's the least important (les important than video streaming). But if there is a significant amount of gaming traffic and crashplan has already lowered down to 1% or so, then the video streaming should start to lower down, because gaming is more important than video streaming.

Is ther really no QoS engine that has simple priorities like this?

I like everything about DD-WRT except that there is only 1 priority class below standard. If everything automatically tagged as exempt, I think I could put video in Express, FTP in standard, and crashplan in bulk, and it would do what I want, but I don't know how to easily put everything in Exempt.

I supposed I could make a ton of port range rules, but I would have to make a range in between every individual port that I need to specify.

Like
1-20 = exempt
21 = standard (FTP)
22-4443 = exempt
4444 = bulk (torrent)
4445-4999 = exempt
5000 = standard (video)
5000-65535 - exempt

Would that be advisable even for gargoyle? It seems silly to have to do that. Though it would be much longer as I am manually specifying about a dozen ports Or is there an easier way to tag "ALL traffic" and then either override the tag for specific traffic to a new tag. Does anyone know if the order in which things are tagged will override other tags?

Or what happens if I were to tag data going to a specific IP as one class, and then say all data going out port 443 goes to another class. What happens to data that matches both rules (data that goes to that IP via 443)?

Re: Question on QoS capabilities

Posted: Tue Nov 05, 2013 5:36 pm
by SirMaster
I am trying to think about how I could do this in Gargoyle.

Lets simplify it and combine CrashPlan, Torrents, and FTP into one class. I could be OK doing that.

Now lets say I set all those to a class and set it's percent bandwidth to 1%

So far so good, when other things use bandwidth, all that file transfer stuff should throttle down to 1% if need be as far as I understand.

Now lets create a class for video streaming. Let's set this class to minimum bandwidth of 80% to guarentee it 4Mbit of my 5Mbit.

Now when someone is streaming at 4Mbit, my other class should throttle down to 20% correct?

Now what happens when people try to play games? Where does all my unclassified traffic go? Ideally I need it to take more bandwidth away from the 1% crashplan class.

Is there a way to make this happen without specifying every single port and figuring out a specific bandwidth for every port/service? I'm perfeclty OK with "everything else" being in that class so long as that class is above both other classes.

Because right now all my traffic is in the "standard class" in DD-WRT including dota and battlefield, etc, and they all never lag, no matter how much bandwidth is bsing used in Bulk. Even if "bulk" is using 4.9Mbit, ive never seen any normal traffic lag. It's as if "bulk" data doesnt even exist. It just gets the hell out of the way of anything else.

What I want is crashplan to get out of the way of video streaming, and video streaming to get out of the way of everything else. If I can at least do that then I think it's worth switching to gargoyle as it will be an improvement over my current setup.

Re: Question on QoS capabilities

Posted: Tue Nov 05, 2013 7:46 pm
by pbix
If DD-WRT is doing exactly what the OP wants to do then why switch?

Gargoyle QoS is not the same a DD-WRT.

Re: Question on QoS capabilities

Posted: Tue Nov 05, 2013 8:03 pm
by SirMaster
I guess I'm not doing a very good job at explaining what I want.

DD-WRT is not doing what I ideally want form my QoS.

I'll paste this part the focuses on my problrm with DD-WRT.

Now onto what I do not like about my current setup.

What I don't like is that Crashplan has to fight with FTP and has to fight with video streaming. Long story short, my parents for example stream video from my server and this takes 4Mbit when they do so. Currently If crashplan is backing up, it and video streaming are both fighting with the 5mbit and each gets about 2.5mbit. This means the video streaming does not work and constantly buffers until I pause crashplan.


I would like to resolve that issue, thus improving my overall QoS without making the parts that I already like worse than they are now.

Basically my setup is only OK now as it only does half of what I want. I'm trying to make it great and do everything I want.

Basically I just want 3 different services to be able to use 99% of my bandwidth each, but with a priority order.

CrashPlan should use 99% of my upload, but only when nothing else is using it (it should always use whatever is left over). FTP should be able to use 99% and take all the bandwidth away from CrashPlan when someone is downloading from FTP, outgoing video streaming should be able to use 99% of my bandwidth and take away from FTP and or CrashPlan (so FTP and CrashPlan never interupt or limit streaming bandwidth), and then normal traffic that I have not classified (or that I can lump into a big classification) should take away from all 3 lower tiers so that web browsing, gaming, skype, and everything else is never impacted by my CrashPlan, FTP, or video streaming.

I'm just trying to get a handle on whether the behavior I want is even possible with Gargoyle before I go through all the hassle of flashing a new firmware and playing with QoS settings for a few days plus setting everything else back up.

Re: Question on QoS capabilities

Posted: Tue Nov 05, 2013 8:21 pm
by pbix
Gargoyle cannot do exactly outline above. If that is all you will accept then there is nothing more that I can do for you.

Gargoyle can make sure that video streaming never pauses and that all WAN bandwidth is utilized when there is a use for it. It an also make sure that some classes get relatively more bandwidth than others. By relative I mean 2x or 4x or 10x but never will it completely cut off a class. To accomplish these things you will have to forget about the DD-WRT way and your way and learn about the Gargoyle way.

Re: Question on QoS capabilities

Posted: Wed Nov 06, 2013 8:03 am
by Wisse
SirMaster, we're running circles here. Think carefully about what pbix and I wrote.
Some specific answers:
What I don't like is that Crashplan has to fight with FTP and has to fight with video streaming. Long story short, my parents for example stream video from my server and this takes 4Mbit when they do so. Currently If crashplan is backing up it and vidoe streaming are both fighting with the 5mbit and each gets about 2.5mbit. This means the video streaming does not work and constantly buffers until i pause crashplan.
You can create (almost) as many classes as you want in Gargoyle. Which means that you can create few "bulk" classes with different bandwidth priority.
But again, you can't completely throttle a class. Let's say you allocate 1% BW to class A. In best case, class A will get 100% of BW and in worst case it's gonna get 1/100 = 1% BW. Say you add class B with 4%. In best case it's gonna get 100% of BW in worst case 4%. If only A in B compete they're gonna get 1*(100/(1+4))/100 = 20% and 4*(100/(1+4))/100 = 80%...
I'm sure you can wrap your head around this and decide if it's suitable for you. Also check my original post where I suggested example configuration for you specific case.
What I like is that it allows me to run as many uploading programs as I want at the same time and not have to worry about figuring out a bandwidth cap for any of them and it never affects any of my normal day-to-day traffic.
It'll be the same with Gargoyle as long your "normal" class (to which unclassified traffic defaults) doesn't run out of BW.
...but I don't know how to easily put everything in Exempt.

I supposed I could make a ton of port range rules, but I would have to make a range in between every individual port that I need to specify.

Like
1-20 = exempt
21 = standard (FTP)
22-4443 = exempt
4444 = bulk (torrent)
4445-4999 = exempt
5000 = standard (video)
5000-65535 - exempt

Would that be advisable even for gargoyle?
You can make "Exempt" class in Gargoyle and default to it. I don't see the need to default to highest class tho. It should be reserved for "special" applications.
I can't say that using port ranges is an advisable way to make QoS rules. You want to have control, which means as strict rules as possible, because that gives you a level of prediction when improving/troubleshooting QoS and BW allocation.

You should take time and make rules for few of your and your roommates' games.
Now lets say I set all those to a class and set it's percent bandwidth to 1%

So far so good, when other things use bandwidth, all that file transfer stuff should throttle down to 1% if need be as far as I understand.

Now lets create a class for video streaming. Let's set this class to minimum bandwidth of 80% to guarentee it 4Mbit of my 5Mbit.

Now when someone is streaming at 4Mbit, my other class should throttle down to 20% correct?

Now what happens when people try to play games? Where does all my unclassified traffic go? Ideally I need it to take more bandwidth away from the 1% crashplan class.
In this case you're left with 19% BW for "normal" class (worst case). Games will lag and pages will load slowly if you'll run out of BW. "Normal" class can't take away from that 1%. Even if it could, it would only be able to take exactly 1%, which means you would get 20% for "normal".
You'll have to make compromise here. I would certainly give video less priority, giving it 80% of BW when "normal" is not active. So for this case (again) you could give Crashplan 1%, video 4% and the rest to normal (this is not how your final QoS should look).

I think you got enough information from us. Please ponder carefully about it before asking the same thing for the 4th time. You're welcome to ask other questions tho.