Min bandwidth not working in 1.5.4

[fightingdreamer]

Theoretical connection speeds: 4/1 MBits/s
Real connection speeds: 4.04/0.81 MBits/s

Wifi is off.

My router is D-Link DIR-300 Rev.A1

Code: Select all

---------------------------------------------------------------
|          _____                             _                |
|         |  __ \                           | |               |
|         | |  \/ __ _ _ __ __ _  ___  _   _| | ___           |
|         | | __ / _` | '__/ _` |/ _ \| | | | |/ _ \          |
|         | |_\ \ (_| | | | (_| | (_) | |_| | |  __/          |
|          \____/\__,_|_|  \__, |\___/ \__, |_|\___|          |
|                           __/ |       __/ |                 |
|                          |___/       |___/                  |
|                                                             |
|-------------------------------------------------------------|
| Gargoyle version 1.5.4    | OpenWrt backfire branch         |
| Gargoyle revision 453676a | OpenWrt revision r30752         |
| Built February 29, 2012   | Target  atheros/default         |
---------------------------------------------------------------

Code: Select all

root@Gargoyle:~# cat /proc/cpuinfo | grep system
system type		: Atheros AR2317

Code: Select all

root@Gargoyle:~# df -h 
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 2.8M      2.8M         0 100% /rom
tmpfs                     6.6M    256.0K      6.4M   4% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/mtdblock3          320.0K    240.0K     80.0K  75% /overlay
mini_fo:/overlay          2.8M      2.8M         0 100% /

Download QoS:

Code: Select all

[Classification Rules]
Match Criteria                                              | Classification
Source       Port:  80,    Connection  bytes:  1024  kBytes   Normal
Source       Port:  443,   Connection  bytes:  1024  kBytes   Normal
Source       Port:  80                                        Fast
Source       Port:  443                                       Fast
Source       Port:  53                                        Fast
Destination  Port:  6800                                      p2p
Destination  Port:  6801                                      p2p
Source       Port:  27960                                     games

Default Service Class: Slow

Code: Select all

[Service Classes]
Service Class Name | Percent BW | Min BW | Max BW | Min RTT | Load
Voip                 1%           80       200      Yes       0.0
Fast                 40%          1000     nolimit            0.8
Slow                 8%           250      nolimit            0.4
Normal               40%          2000     nolimit            197
p2p                  1%           zero     nolimit            3924
games                10%          500      nolimit  Yes       0.0

Total Download Bandwidth: 4000 kbit/s

Code: Select all

[Active Congestion Control]
Congestion Control Status
State:             ACTIVE
Link limit:        3967 (kbps)
Fair Link limit:   4000 (kbps)
Link load:         4143 (kbps)
Ping:              101 (ms)
Filtered ping:     89 (ms)
Ping time limit:   98 (ms)
Classes Active:    2

Upload QoS is also active[similar classes and classification rules], only "Max BW" for p2p.

Code: Select all

Total (Upload) Bandwidth: 820 kbit/s

I checked tab "Status->Connection List", and all connections are marked to appropriate classes.

The above settings are captured during the test:
I started one http download: http://noc.gts.pl/500mb.gts ,
waited until "Link load" hit 3970 (kbps), in class "Normal" the Load also hit ~3970,
started torrent with many seeds[without DL speed limit, UL limit to 10% UL total speed],
waited ~60sec, ~120sec, ~300sec, captured above settings[from second ~43 almost nothing changed].

I used the same settings on 1.4.6 and "Min BW" always working, cutting the "p2p" load.
It seems that Min BW not working for some reason on 1.5.4, not only for stock "Normal" class, but also for custom classes.

Before test I resets all settings by "System->Backup/Restore->Restore Default Configuration",
reboot router by "System->Reboot", waited until router turns on and I'll be able to establish a connection, set minimal config, waited 5 min[to be sure that internal processes are completed, I did not want to login via ssh and check].
I did test,
reboot via power off > waited 5sec > power on, waited 5 min after established a connection,
did the test again and again.

Second and third tests give almost the same result[few kbits difference] as first.

Do Qos Min bandwidth really does not work on 1.5.4? and I have to go back to 1.4.6/1.4.7? or may need an additional set something in 1.5.4?

If additional informations will be needed, ssh works and I can log on.

[pbix]

Nice report fightingdreamer, versions used mentions, router used mentioned, good explanation of what you did.

According to your report neither MinBW or %BW are working properly since by %BW the normal class should have 40x the p2p class.

In response to your report I tried a test on my router which is a Buffalo WZR-HP-G300NH running v1.5.4. I found that both MinBW and %BW are working properly. So I have not reproduced your result.

Looking at why.

First, I do not do torrenting and instead relied on large file downloads. I used the one you suggested as well as a download of http://www.ubuntu.com/download/desktop. I made two classes and a rule with 10MB "Connection bytes reach". Start one, when it changes class due to the rule start the other and see if the sharing is correct.

Second, your router has only 16MB of RAM which is rather constrained. QoS is setup slightly different in 16MB routers to try and prevent problems. You can eliminate this difference by editing the file /etc/init.d/qos_gargoyle on your router. Look for the string "depth 32" and change it to "depth 128". Save and try again with your torrent traffic.

Finally, before you make any changes please run the following from the command line of your router.

/etc/init.d/qos_gargoyle retart > /tmp/savefile
/etc/init.d/qos_gargoyle show >>/tmp/savefile

There is a significant difference between the design of 1.4.6 and later releases. See http://www.gargoyle-router.com/phpbb/vi ... =14&t=2353 for details. The root of this problem is probably related to that.

Post the result as an attachment to your next post.

[fightingdreamer]

First I had to use a failsafe and run firstboot, because something screwed up by changing QoS settings via ssh[before you responded to my post].

In D-Link DIR-300 Rev.A1 is needed to wait exacly 55sec from power on, to see the failsafe packet in wireshark.

Entry into failsafe is very easy:
power on -> wait 53 sec -> press reset button -> wait 4 sec -> release reset button

I set the previous settings, only had to change p2p ports to 10117-10118, also checked whether packets get into the appropriate classes.

------------------------------------

/etc/init.d/qos_gargoyle retart > /tmp/savefile
"retart" is not a valid argument for /etc/init.d/qos_gargoyle, so I used the closest similar "restart".

Code: Select all

root@Gargoyle:~# /etc/init.d/qos_gargoyle restart > /tmp/savefile

gave some output but /tmp/savefile is empty:

Code: Select all

root@Gargoyle:~# wc -l /tmp/savefile 
        0 /tmp/savefile

Code: Select all

root@Gargoyle:~# /etc/init.d/qos_gargoyle show >>/tmp/savefile
root@Gargoyle:~# wc -l /tmp/savefile 
      205 /tmp/savefile

copied the file:

Code: Select all

ssh root@192.168.1.1 'cat /tmp/savefile' | cat > /tmp/savefile

Yes my router has only 16MB of RAM, but when heavy load is used only 82% of RAM.

------------------------------------

I changed /etc/init.d/qos_gargoyle as you wrote:

before:

Code: Select all

root@Gargoyle:~# grep -n 'depth [0-9]' /etc/init.d/qos_gargoyle -A1 -B1
321-	if [ "$total_mem" -lt 16000 ] ; then
322:		esfq_depth="depth 32";
323-	else
324:		esfq_depth="depth 128";
325-	fi

now:

Code: Select all

root@Gargoyle:~# grep -n 'depth [0-9]' /etc/init.d/qos_gargoyle -A1 -B1
321-	if [ "$total_mem" -lt 16000 ] ; then
322:		esfq_depth="depth 128";
323-	else
324:		esfq_depth="depth 128";
325-	fi

After I changed the depth to 128:

Code: Select all

/etc/init.d/qos_gargoyle restart

have done tests.

Urls used:
[direct] http://noc.gts.pl/500mb.gts
http://www.ubuntu.com/download/desktop

I downloaded by:

Code: Select all

wget 'url' -O - > /dev/null

Torrent used in test:
http://torrent.ubuntu.com:6969/
bbb6db69965af769f664b6636e7914f8735141b3

QoS started to work but very unstable, a single http connection speed floating between ~8KBytes/s and ~440KBytes/s[random time between rising/falling/stable speed], after 5min still unstable.

I also tried to change the ping limit for the lower value[from Auto to 10ms], this worked a little better, still unstable but http connection never gone down below 40KBytes/s.
The problem is because I can not change second ping limit, I prefer a shorter response time than high transfer.

Setting of all classes as the RTT, to force first ping limit is probably a bad idea?

Also I caught this during the test:

Code: Select all

Congestion Control Status
State: ACTIVE
Link limit: 3827 (kbps)
Fair Link limit: 3796 (kbps)
Link load: 5651 (kbps)
Ping: 91 (ms)
Filtered ping: 103 (ms)
Ping time limit: 99 (ms)
Classes Active: 2

"Link load" is much greater than "Link limit".

[pbix]

What kind of WAN connection do you have? Is DSL with bridged modem?

It looks like maybe you are doing the file downloads on your router with wget? If so, please switch to using a separate computer on you LAN to do these downloads.

Please disable the ACC on your router and set the download speed to 3Mbps.

We need to simplify the setup to establish exactly what is causing your result. After we get my result reproduced on your setup we will see what it takes to break it.

What CPU load is shown on the status screen while this download is going on? Is the CPU load nearing 1.0?

Sorry about the 'restart' typo in my previous post. But I cannot understand why you cannot pipe the output of a qos_gargoyle restart to a file. Please try again. There should be several pages of output. If you are not getting any output something is wrong.

[fightingdreamer]

WAN connection: just a ethernet cable[rj-45], about 80meters from ISP switch.

About wget, I using Linux, wroted about it because some download programs incorrectly show the download speed[sorry about that].

Of course I use the computer in LAN, no downloads on the router.

Reverted value depth to 32 in /etc/init.d/qos_gargoyle.
Reverted ping limit to Auto.

I tried again:

Code: Select all

root@Gargoyle:~# /etc/init.d/qos_gargoyle restart > /tmp/savefile

gave some output but /tmp/savefile is empty:

Code: Select all

root@Gargoyle:~# wc -l /tmp/savefile
        0 /tmp/savefile

So redirected stderr to file:

Code: Select all

root@Gargoyle:~# /etc/init.d/qos_gargoyle restart 2> /tmp/savefile
root@Gargoyle:~# wc -l /tmp/savefile 
       92 /tmp/savefile

root@Gargoyle:~# /etc/init.d/qos_gargoyle show >>/tmp/savefile
root@Gargoyle:~# wc -l /tmp/savefile 
      297 /tmp/savefile

5 minutes from the start torrent and http single connection.

Code: Select all

CPU Load Averages: 0.52 / 0.19 / 0.13  (1/5/15 minutes)

I will try to disable the ACC, set download speed to 3Mbps, and see how it will work.

_edit: QoS started to work, immediately cut p2p traffic.
After 5min still stable:

Code: Select all

[Service Classes]
Service Class Name | Percent BW | Min BW | Max BW | Min RTT | Load
Voip                 1%           80       200      Yes       0.0
Fast                 40%          1000     nolimit            0.0
Slow                 8%           250      nolimit            207
Normal               40%          2000     nolimit            2627
p2p                  1%           zero     nolimit            66
games                10%          500      nolimit  Yes       0.0

[pbix]

It would be beneficial to other users if you could report what exactly was required to make your QoS start working again. It is not clear from your previous post what that was.

[fightingdreamer]

I did what you told:
Disabled ACC and set DL speed limit to 3MBits/s.

ACC is still not working[because it's off], but classes based priority works.
So the problem is with ACC.

I thought this part would be obvious:

I will try to disable the ACC, set download speed to 3Mbps, and see how it will work.

_edit: QoS started to work, immediately cut p2p traffic.
After 5min still stable:

[pbix]

Well it could be that you reduced your download speed I guess.

Now tell me about your WAN connection. What is the ping target that ACC is using? Is it on the ISP side of your link? Make sure it is.

Then turn the ACC on in Auto mode. Post the ACC status section while it is operating.

[fightingdreamer]

Ping Target: 192.168.0.1

traceroute:

Code: Select all

$ mtr 208.67.220.220 -c1 -r -n
HOST: xxx-desktop                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1.|-- 192.168.1.1                0.0%     1    1.6   1.6   1.6   1.6   0.0
 2.|-- 192.168.0.1                0.0%     1    9.1   9.1   9.1   9.1   0.0
 3.|-- 193.39.x.x                 0.0%     1    8.0   8.0   8.0   8.0   0.0
 4.|-- 10.10.33.1                 0.0%     1    6.5   6.5   6.5   6.5   0.0
 5.|-- 193.39.x.x                 0.0%     1   42.1  42.1  42.1  42.1   0.0
 6.|-- 46.28.244.13               0.0%     1   35.8  35.8  35.8  35.8   0.0
 7.|-- 82.112.96.21               0.0%     1   18.5  18.5  18.5  18.5   0.0
 8.|-- 129.250.4.77               0.0%     1   38.7  38.7  38.7  38.7   0.0
 9.|-- 129.250.2.161              0.0%     1   36.1  36.1  36.1  36.1   0.0
10.|-- 129.250.2.122              0.0%     1   76.1  76.1  76.1  76.1   0.0
11.|-- 213.198.52.82              0.0%     1   43.3  43.3  43.3  43.3   0.0
12.|-- 208.67.220.220             0.0%     1   37.6  37.6  37.6  37.6   0.0

192.168.1.1 is D-Link DIR-300 with gargoyle firmware
192.168.0.1 is ISP main router

You do not see this in traceroute, but between 192.168.1.1 and 192.168.0.1 there is at least one switch, belonging to the ISP.

ACC:

Code: Select all

[Active Congestion Control]
Congestion Control Status
State:             ACTIVE
Link limit:        2971 (kbps)
Fair Link limit:   3000 (kbps)
Link load:         3019 (kbps)
Ping:              4 (ms)
Filtered ping:     6 (ms)
Ping time limit:   102 (ms)
Classes Active:    2

----------------------------

Also tried to set "Use non-standard ping target: 192.168.0.1" and the DL speed limit back to 4000.
When I set it, the QoS stopped working properly, so I reverted this changes.

Whether it is possible to generate statistics via ssh, such as tab QoS (Download/Upload)?
I got the idea to collect the statistics and present them in a graph of time.

[pbix]

Also tried to set "Use non-standard ping target: 192.168.0.1" and the DL speed limit back to 4000.

I think said 192.168.0.1 is your standard ping target. It this a typo above when you set an alternate ping target of the same 192.168.0.1? Is there a reason for this?

I think the problem here is the ping target. Gargoyle really has no good way to do pick it other it than just pick the upstream router. On a complicated WAN like yours this is too simplistic. One of the hops shown in the tracert is the link most subject to congestion. We need to target the router on the other side of that link.

So let's set your DL speed to 4500 and start working up the chain with the ping target. You already tried the first two, just try 3 thru 12 until the ACC starts working correctly. When you have the correct one is suspect your ping times will be in the 50-100ms range and not 4ms as you have now.

You will know that ACC is working correctly when the link limit automatically moves to under 4000 when you are downloading a large file. Don't worry about if QoS is "working" right now. Lets just get the ACC working, QoS will follow.

What do you know about the details of your ISP and their connection to the Internet? It would be interesting to know.


Please post your ACC status sections as you go. I want to see the ACC in Active mode with the link in saturation.