QOS For Router Not Working

[pbix]

It has been the goal of Gargoyle QoS to apply the rules stated on the QoS pages to the WAN traffic destined for the router itself as well as traffic passing through the router. In particular the qos_ingress chain is targeted in the INPUT mangle table. Only traffic destined for the router itself traverses this table.

So I think I have determined that this cannot work. The problem is that the IMQ device which is used for ingress traffic control can only be entered in the PREROUTING and POSTROUTING tables. Packets destined for internal router processes get marked and selected to go into IMQ0 but since this selection is done in the INPUT table they don't actually go into the IMQ device and instead just go straight to the process.

Traffic which originates in the router and goes out the WAN interface (egress) is handled by QoS as expected. Traffic going through the router also works. It is only the incoming WAN traffic going to the router itself that this post applies to.

So I have read and studied this in some detail and I do not see any easy way to resolve this issue. We cannot use the PREROUTING table because the mangle chain is transversed before NAT is done. We cannot use the POSTROUTING table because the traffic in question never hits this table.

If anyone has any bright ideas now would be a good time.

[Eric]

Errr... both the FORWARD and INPUT chains jump to qos_ingress, which causes traffic to pass to IMQ. See lines 590-597 of the init script:

Code: Select all

# Mark ingress in FORWARD and INPUT chains to make sure any DNAT (virt. server) is taken into account
iptables -t mangle -A FORWARD -i $qos_interface -j qos_ingress
iptables -t mangle -A INPUT -i $qos_interface -j qos_ingress
 	
apply_all_rules "download_rule" "$class_mark_list" "qos_ingress" "mangle" "$download_mask"

#pass traffic through IMQ
iptables -t mangle -I qos_ingress -j IMQ --todev 0

The INPUT chain only goes to the router, while everything else goes in FORWARD, and they both lead to IMQ.

[pbix]

Lets confine our discussion to the INPUT chain for now.

I agree that there are mangle rules that target the packets to IMQ. But the trouble is that packets can only actually be moved to IMQ immediately after the PREROUTING or POSTROUTING tables.

To see what I am talking about run ping on the router
ping -c 1000 http://www.google.com

inspecting the mangle chain
iptables -vnL -t mangle

You can see that the ping packets are properly marked and targeted to imq0.

Now look at the imq0 class packet counts
tc -s class show dev imq0

You would expect the ping responses incrementing the packet counts as well. But you will see that there are no ping packets actually arriving in any of the classes of imq0. You need to have a quiet LAN to see this since you don't want LAN traffic to confuse you.

These packets are not going to imq0. They are going directly to the ping process and bypassing QoS.

A good site to study to see what is going on is
http://www.docum.org/docum.org/kptd/

[Eric]

Ah... yes. Good point, there is a problem.

But there is a solution: we need to restore the mark from the connmark in the ingress PREROUTING chain. The first packet of a connection won't get classified properly, but the second one will, since the connmark (which is persistent) will be set once that first packet its the qos ingress chain.

[pbix]

Your idea is to leave the mangle chain rules as is in the INPUT table except for the connmark restore? The idea is to move only the connmark restore to the mangle chain of the PREROUTING table?

If I have it correct I will check your idea out this weekend.

[Eric]

Yes. The connmark gets set after IMQ so the first packet doesn't get classified properly, but for subsequent packets in the connection the connmark will be set and a simple connmark restore in the PREROUTING chain should set the packet's mark to the connmark before it gets to IMQ.

[pbix]

Eric,
OK I think I can confirm that your approach could work but it will force QoS to depend on connection tracking which is something it does not do now (expect for layer7 marking). Not a big deal I guess but as a result all non-connection traffic like ICMP will not get marked and will fall into the default class. Also handling this issue will introduce some complexity that we do not now have.

The interesting part is how does a user specify the router IP address in a classification rule. There is no way to do that now. I suppose I could allow him to 127.0.0.1 or the router address (ie 192.168.0.1) as the source/dest address. That could be managed in qos_gargoyle but in I have observed that such rules must only appear in the INPUT and OUTPUT tables since there is no other good way to isolate this traffic.

So we would have the following QoS chains
qos_ingress_imq in PREROUTNG (gets traffic into the IMQ)
qos_ingress_lan in FORWARD (ingress bound for LAN)
qos_ingress_host in INPUT (ingress bound for router)
qos_egress_lan in FORWARD (egress from LAN->WAN)
qos_egress_host in OUTPUT (egress from router->WAN)

Of course the other approach is to just leave it alone and say we do not do QoS for router traffic. Do we know of any significant source of router generated traffic? I suppose some command line commands like "opkg update" or some other package might cause it. Also with DD-WRT I saw users running torrents on their router so as it is right now we cannot handle that.

I suppose I will code up the above five chain idea and see how bad is looks.

[Eric]

Let's only worry about one problem at a time. For now don't worry about router-specific traffic. I'm not clear on how/why this isn't working (matching the router's WAN IP should work), but it's a minor point.

You're right that ICMP isn't going to get classified properly, but there's no reason there should be significant ICMP traffic on the network. By far the most common type of traffic is TCP, which this approach will handle just fine. Ideally we'd like to handle this, but it may not be possible.

The fix should be simple -- just create a chain with one rule in PREROUTING that all ingress packets (on the wan interface) hit that contains one rule which restores the connmark to the mark. That's a total of three chains -- we have two now (though the ingress chain does get accessed from both the INPUT and FORWARD chains). I can do it if you'd prefer, but you've been doing a good job of maintaining the QoS lately (great job on identifiying this bug by the way!)

I don't understand why you think we should have separate ingress and egress chains for INPUT/FORWARDING/OUTPUT. Egress is working fine -- the above problem only affects ingress traffic.

[pbix]

Let's only worry about one problem at a time. For now don't worry about router-specific traffic.

If we "don't worry about router-specific traffic" then there is nothing to do. The current v1.2.3 handles non router-specific traffic correctly. You've confused me now.

I'm not clear on how/why this isn't working (matching the router's WAN IP should work)

I get my WAN address via DHCP from my ISP. There is no static WAN IP that I can use in rule writing. We would need some other handle (like 127.0.0.1)

The fix should be simple

A fix to handle both router and LAN generated traffic can be done and I will do it. Not sure I would call it "simple" however but its not really that bad either. The issue of not knowing what the WAN IP is at the time of rule writing is why the additional chains are needed. I've already started coding some stuff. its not really that bad.

[Eric]

Ah... ok. Here's what was confusing me, and makes the issue a bit tricky:

Right now we're entering IMQ0 in the INPUT and FORWARD chains, but not the PREROUTING chain, (though PREROUTING is the typical configuration). I originally took this configuration from elsewhere (we need to classify packets after NAT so this is appealing), and it seemed to work, but there is essentially no documentation on doing this. I thought you had come to the conclusion that this whole configuration with IMQ in INPUT and FORWARD chains couldn't work.

Given this, my thought was to pass ingress packets to IMQ in PREROUTING as normal, and restore the marks there, evading this strange / poorly documented configuration.

But it looks like for some reason this configuration works with the FORWARD chain, but not the INPUT chain. Knowing the correct WAN IP address isn't the issue here -- they would end up in the default IMQ class if it was just a problem with classification. What's happening is that for some reason packets in the FORWARD chain are going through IMQ, but packets in INPUT are not. I'm not sure if moving IMQ passthrough to PREROUTING and restoring from the connmark is worth the tradeoff (given that it's working in FORWARD), just to handle INPUT packets. What do you think?