Graceful way to split Dual Access traffic?

[eierfrucht]

Hi!

I'm running an Asus RT-N16 with custom firmware as a kind of L2TP modem (takes LOTS of CPU power to sustain a 40 Mbps L2TP tunnel at top load), one of its LAN ports bridged with the WAN port of my home gateway, TP-LINK 1043ND running Gargoyle 1.5.11.

My ISP down / up speeds have been perfectly stable since ages, i.e. I don't need active congestion control. I'm getting some 40-42 Mbs via L2TP and another 60 Mpbs (100 Mbps MAN link minus the 40 Mbps max L2TP capacity) between me and my ISP's middle area network.

What I want is to isolate MAN traffic as a separate traffic class: anything that goes to or from 10.0.0.0/8 should be limited to 60% (60 Mbps) of total link capacity at a minimum bandwidth of 60 Mpbs and maximum bandwidth of 60 Mbps.

Anything that doesn't classify as coming to or from 10.0.0.0/8 should be classified by one of the further rules in the chain, all of which add up to some 40% of remaining max bandwidth.

The question is: how exactly does Gargoyle judge if the link is saturated?

Imagine a situation where there is absolutely no MAN traffic, yet a minimum of 60 Mbps is reserved for it. Meanwhile, the L2TP tunnel on my RT-N16 is overloaded, and there are some outbound packets on Gargoyle's side waiting to be sent because the RT-N16 just can't squeeze them all through.

Will Gargoyle consider the 60 Mbps _minimum_ reserved for MAN traffic (though not actually used) as effectively contributing to link saturation and think like "Hey, we've already got a minimum of 60 Mbps reserved for Class A and the rest of classes are currently producing a total 40 Mpbs of actual traffic, so the link is saturated and i must start balancing the traffic between those classes without a strict minimum and / or maximum limit, yet assigned with a total percentage limit!"

Or rather the unused (yet reserved as a _minimum_) 60 Mpbs of MAN bandwidth will delude Gargoyle into thinking that the link is not saturated, so it won't take any action towards shaping the L2TP traffic, thinking that the link is only 40% (40 Mbps) saturated?

I could simply exempt the local MAN traffic with an iptables script, but that is likely to jeopardize my WAN speeds -- a massive data transfer coming over MAN (and ignored by QoS) would easily choke the MAN link itself through which the L2TP tunnel runs, thus squeezing the L2TP connection itself.

I'm also not enthusiastic about limiting my overall link to 40 Mbps just to comply with L2TP speeds.

Any help appreciated and sorry for the occasionally bad English.

[pbix]

Gargoyle QoS does not "reserve" bandwidth and the link is saturated with the link load reaches the link limit you put in on the QoS pages.

Please read the Wiki http://www.gargoyle-router.com/wiki/doku.php?id=qos

If you still have question post them here and I will address them.

[eierfrucht]

So, there seems to be no way to make QoS work with a specified downlink of 100 Mbps when the 40 Mbps L2TP tunnel on the outer router becomes overloaded, but there still remains 60 Mbps or so of unused MAN bandwidth?

(Provided that I can easily capture MAN traffic in a class of its own by adding a "Destination: 10.0.0.0/8" rule on top of the rule chain)

I've already read the QoS wiki but was a bit unsure how the minimum bandwidth limit worked in terms of link saturation. If a class is given a 40 Mbps minimum bandwidth, wouldn't that mean that as long as the WAN link is up, at least 40 Mbps of its bandwidth counted towards the overall link load?

It sounds natural that if a class is given a _minimum_ bandwidth, that bandwidth will never be shared with any other classes... isn't the minimum bandwidth limit supposed to guarantee a class some bandwidth in a way so it won't get shared across other classes under any circumstances?

Is there a way to simultaneously:

1. Specify a down / up link of 40 Mpbs in the QoS settings.

2. Make all traffic to and from 10.0.0.0/8 absolutely exempt from any QoS regulation.

3. Limit this exempt traffic to a total bandwidth of, say, 50 or 60 Mpbs, so it never starts choking the L2TP (WAN) bandwidth. Like 100 Mbps of total MAN bandwidth minus 40 Mbps of L2TP (WAN) bandwidth makes 60 Mpbs of free bandwidth, of which I can safely utilize at least 50 Mbps for local MAN traffic without worrying about pushing my L2TP (WAN) bandwidth below the limit specified in QoS settings. My L2TP speeds are very stable around 42-43 Mbps 24/7, provided that there's no local MAN traffic.

4. Keep these changes active even after some changes are done via GUI.

So far I was only able to find a solution for 1. http://www.gargoyle-router.com/phpbb/vi ... 2&start=10

It's flawed though -- the changes are instantly gone once you edit QoS optons via GUI, secondly there's no bandwidth limit on the MAN traffic -- which makes it easy for MAN traffic to choke the WAN traffic because the two share the same physical link -- thus ruining the whole QoS thing as long as ACC is disabled.

[pbix]

Again, Gargoyle does not 'reserve' bandwidth so its clear you are struggling with the concepts and you need to step back, enjoy a glass of beer and ponder.

When the link is not saturated all classes are limited only by their maximum bandwidth setting. So in those questions above in which the link is not saturated (that is at 100Mbps in your case) there will be no effect from any QoS class (since you are not using the maximum bandwidth setting).

There is a door to/from the internet, when a packet arrives to your router if there is no other packet waiting it just goes through the door. That is all. If there are packets waiting it means the link is saturated. Waiting packets is the definition of link saturation.

Now we know that we only need to think about the link saturation cases and design our QoS for those cases.

I think I understand that you have two routers and the one running the L2TP tunnel is the one connected to the ISP and it is not running Gargoyle. Is that correct? How is the L2TP limited to 40Mbps then?

Can you connect your Gargoyle router to your ISP and connect your L2TP router to one of the LAN ports of the Gargoyle router? That would give us the most control and the best chance of satisfaction.

If you cannot make any changes then we can use the ACC, it may provide satisfactory results for you. I will show you how after after we clear up the above.

There is no "exempt" concept in Gargoyle QoS and from what I understand so far such a concept would not help you anyway.

[eierfrucht]

This question should probably have been asked in the first place, but better late than never... is it possible to make one of Gargoyle's LAN ports into a second WAN with a different WAN IP, then head all 10.0.0.0/8 traffic through this second WAN? If possible, I'll just filter all such traffic on RT-N16 by Source IP, route it over MAN and limit it to 60 Mbps max with RT-N16's TomatoUSB QoS. The rest of the traffic will pass through Gargoyle's primary WAN, meet the 40 Mbps QoS limit, be subject to Gargoyle's QoS regulations, etc. Problem solved?

I think I understand that you have two routers and the one running the L2TP tunnel is the one connected to the ISP and it is not running Gargoyle. Is that correct?

Right you are. We have a dual access system here. The RT-N16 running TomatoUSB gets its MAN address from the ISP over DHCP. My ISP's Middle Area Network spans a huge area, yet it has no direct access to the web. I mainly use it for extra bandwidth between me & friends that costs me no web traffic.

Every time the RT-N16 boots, it sets up a L2TP tunnel between it and my ISP's L2TP server, which is accessible from MAN. Once the L2TP tunnel is up, it acts as WAN. Any data that is transmitted between the RT-N16 and the internet consumes my MAN link, because the L2TP tunnel itself runs over MAN. Still, local MAN connections don't need L2TP, but they compete with L2TP for bandwidth.

My MAN link limit is 100 Mbps, my L2TP bandwidth limit is 40 Mbps.

I want to 'cap' my remaining MAN bandwidth at 60 Mbps so that the local MAN traffic never competes for bandwidth with the L2TP traffic but I'm able to use it for local MAN connections.

The Gargoyle's (1043ND) WAN port is simply connected to one of the RT-N16's LAN ports. Gargoyle is what I use for a home gateway, mainly because of its much more versatile QoS and superior wireless capability -- RT-N16 is Broadcom-based and, as such, sometimes starts acting funny in WDS mode, other times the wireless just freaks out for no apparent reason, regardless of firmware brand and version. You know those proprietary Broadcom drivers don't you. I still desperately need my RT-N16 because it's powerful enough to sustain a 40 Mbps L2TP tunnel and supports a variety of firmwares with highly efficient L2TP support, like Oleg's or TomatoUSB.

My current QoS settings for Gargoyle are very basic: 39000 Mbps total downlink, 39000 Mbps total uplink, Fast Class for inbound / outbound ports 80 (< 1024kb) / 443 (< 1024 kb) / 123 / 53 / 37 / etc. at 99% of maximum bandwidth; Slow Class for everything else at 1% of maximum bandwidth.

This helps me a lot: torrents, youtube, http and ftp downloads, and whatever else typically consumes massive traffic but can always wait -- does in fact wait because it always gets only 1% of bandwidth every time there are some packets waiting and 'competing for entry'. This doesn't prevent this class from taking as much bandwidth as possible when the WAN link is not congested.

Web browsing, DNS requests, etc. are getting 99% of bandwidth at times of congestion; while this class is inactive, though, the Slow class happily consumes all available bandwidth -- i.e. when I'm away from keyboard and no background services are currently using the Fast class ports.

This has been helping me a lot; unlike the rigid QoS of TomatoUSB. The drawback is that both MAN and WAN traffic are affected by QoS Limitations, and I only get 40 Mbps for WAN + MAN. Any MAN traffic thus competes with WAN traffic, though in theory I can utilize up to 60 remaining Mbps of free MAN bandwidth.

Can you connect your Gargoyle router to your ISP and connect your L2TP router to one of the LAN ports of the Gargoyle router?

Of course I can connect Gargoyle directly to my ISP then connect my RT-N16's WAN port to Gargoyle's LAN port. Then I will even be able to establish an L2TP tunnel from RT-N16 over Gargoyle and further over the ISP's middle area network all the way to the ISP's L2TP server, but then Gargoyle will only see a single massive L2TP connection coming from its LAN client to a remote server on the MAN/WAN side. In this case, Gargoyle won't be able to execute QoS regulations over the various connections running through the L2TP tunnel, which makes the idea a bit silly as I understand it.

TomatoUSB, however, allows for easy VLAN setup. I wonder if it's possible to split one LAN port of RT-N16 into a separate VLAN with the same subnet as Gargoyle's LAN subnet, then connect this port with a cross cable to one of Gargoyle's LAN ports, then somehow bridge this VLAN with RT-N16's primary LAN via TomatoUSB. In such a way, RT-N16's LAN (which is Gargoyle's WAN) will be reachable in two ways:

1. Simply through Gargoyle's WAN (and through QoS)

2. Over 'software bridge' connecting the two VLANs at RT-N16, because the secondary VLAN has the same subnet as ( and is physically cross-cabled to, and apparently is part of ) Gargoyle's LAN. Such packets will land on Gargoyle's WAN side without having to pass through its WAN port. Actually a MultiWAN of sorts with the help of some cross cable and VLAN settings.

Then:

1. Add some routing to Gargoyle so any packets with 10.0.0.0/8 dest/src reach RT-N16's LAN via the VLAN bridge at RT-N16 itself, not through Gargoyle's WAN port.

2. Add some routing to RT-N16 so any packets with 10.0.0.0/8 dest/src are sent to the MAN gateway, not WAN gateway (this has always been working for me)

Then, finally, configure RT-N16's limited Qos to split all traffic in two classes: 60 Mbps for packets to/from 10.0.0.8, 40 Mbps for everything else.

Then just keep the 40 Mbps QoS limit on Gargoyle and use it for prioritizing web traffic, because all packets headed for MAN will never pass through Gargoyle's WAN port and the 40 / 60 splitting will be done by RT-N16 itself.

[pbix]

This question should probably have been asked in the first place, but better late than never... is it possible to make one of Gargoyle's LAN ports into a second WAN with a different WAN IP, then head all 10.0.0.0/8 traffic through this second WAN?

No Gargoyle has only one WAN port.

If you have specific other concise questions I will respond.

[eierfrucht]

Well, I was able to create a secondary VLAN on my RT-N16 and assign it the same subnet as Gargoyle's LAN. Then I cross-cabled the both and now it works as a kind of multi-WAN for Gargoyle (by default different VLANs on Shibby's TomatoUSB communicate between each other freely) Quite expectedly, QoS only affects traffic leaving/entering Gargoyle through its native WAN port.

Figuring out the proper routing to send 10.0.0.0/8 traffic through the `back door`, for now.

My remaining question is whether a more straightforward approach is possible, without configuring multiple VLANs:

1. Specify a down / up link of 40 Mpbs in the QoS settings.

2. Make all traffic to and from 10.0.0.0/8 absolutely exempt from any QoS regulation.

3. Limit this exempt traffic to a total bandwidth of, say, 50 or 60 Mpbs, so it never starts choking the L2TP (WAN) bandwidth. Like 100 Mbps of total MAN bandwidth minus 40 Mbps of L2TP (WAN) bandwidth makes 60 Mpbs of free bandwidth, of which I can safely utilize at least 50 Mbps for local MAN traffic without worrying about pushing my L2TP (WAN) bandwidth below the limit specified in QoS settings. My L2TP speeds are very stable around 42-43 Mbps 24/7, provided that there's no local MAN traffic.

4. Keep these changes active even after some changes are done via GUI.

So far I was only able to find a solution for 1. viewtopic.php?f=5&t=1722&start=10

It's flawed though -- the changes are instantly gone once you edit QoS optons via GUI, secondly there's no bandwidth limit on the MAN traffic -- which makes it easy for MAN traffic to choke the WAN traffic because the two share the same physical link -- thus ruining the whole QoS thing as long as ACC is disabled.

What I need is some help to elaborate on the existing solution (http://www.gargoyle-router.com/phpbb/vi ... 2&start=10) for exempting all MAN traffic. I need to boost it with 'hard-cap at 60 Mbps' and 'keep changes even after a GUI action is taken' capabilities. Then I'll be able to leave my overall QoS limit at 40 Mbps and handle the MAN / WAN traffic differences on the RT-N16.

I don't like the multiple VLAN solution because it seems just too complicated.