Experimental Gargoyle 1.15.x (released as 1.14) beta 20230124

[ispyisail]

Alright, looks like we're back in action.
TFTP recover using the images available and you should be up and running.

Success

Note: You need to wait for ages for it to finish booting. I just about called it a brick but I thought I had better wait some more.

What was wrong?

[Lantis]

A package (swconfig) was not being pulled in by the devices that needed it.
This would have affected anything not migrated to the DSA switch arch, but only ath79 where this mistake happened.
The router was actually working, but the LAN switch was not being set up correctly, hence no access.

Many thanks to yourself for flashing the test images and in particular pythonic who spent the weekend flashing lots of different builds and hooking up a serial console to understand what was happening.

[ispyisail]

A package (swconfig) was not being pulled in by the devices that needed it.
This would have affected anything not migrated to the DSA switch arch, but only ath79 where this mistake happened.
The router was actually working, but the LAN switch was not being set up correctly, hence no access.

Many thanks to yourself for flashing the test images and in particular pythonic who spent the weekend flashing lots of different builds and hooking up a serial console to understand what was happening.

Thanks, Lantis and pythonic

Do you still want the router?

It really is at the end of life

[Lantis]

I think my wife would kill me if I brought another one into the house
I appreciate the offer!

[tob99]

Intel i350 is not available - i think it's because of missing drivers.

Looks like i350 requires the igb driver which isn't currently built.

Re the Realtek NICs:

1) while you're running a speed test through the device, ssh into it and use top to monitor CPU stats - looking to see whether there might be an issue with CPU maxing out (e.g. due to interrupt load).

Thank you for your answer. I did as you've said and used 'top' command. If i understand this code well there is nothing special in it and cpu seems not to be overloaded.

Code: Select all

Mem: 74880K used, 7900996K free, 480K shrd, 1208K buff, 11288K cached
CPU:   0% usr   0% sys   0% nic  86% idle   0% io   0% irq  13% sirq
Load average: 0.00 0.00 0.00 2/92 8944
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
 1997     1 root     S     1472   0%   0% /sbin/rpcd -s /var/run/ubus/ubus.sock -t 30
 8944  5183 root     R     1136   0%   0% top
 5182  2202 root     S      956   0%   0% /usr/sbin/dropbear -F -P /var/run/dropbear.1.pid -p 22 -K 300 -T 3 -2 9
 8808     2 root     IW       0   0%   0% [kworker/u8:1-ev]
 2516     1 root     S     3564   0%   0% /usr/sbin/uhttpd -f -h /www -r OpenWrt -x /cgi-bin -u /ubus -t 60 -T 30 -k 20 -A 1 -n 3 -N 100 -R -p 0.0.0.0:80 -p [::]:80 -C /etc/uhtt
 8286     1 root     S     2160   0%   0% {dnsmasq} /sbin/ujail -t 5 -n dnsmasq -u -l -r /bin/ubus -r /etc/TZ -r /etc/dnsmasq.conf -r /etc/ethers -r /etc/group -r /etc/hosts -r
 2903     1 root     S     2160   0%   0% {ntpd} /sbin/ujail -t 5 -n ntpd -U ntp -G ntp -C /etc/capabilities/ntpd.json -c -u -r /bin/ubus -r /usr/bin/env -r /usr/bin/jshn -r /us
 2318     1 root     S     1352   0%   0% /sbin/netifd
    1     0 root     S     1236   0%   0% /sbin/procd
 2377     1 root     S     1200   0%   0% /usr/sbin/odhcpd
 8290  8286 dnsmasq  S     1200   0%   0% /usr/sbin/dnsmasq -C /var/etc/dnsmasq.conf.cfg01411c -k -x /var/run/dnsmasq/dnsmasq.cfg01411c.pid
 5183  5182 root     S     1128   0%   0% -ash
 2923  2903 ntp      S     1120   0%   0% /usr/sbin/ntpd -n -N -S /usr/sbin/ntpd-hotplug -p 0.openwrt.pool.ntp.org -p 1.openwrt.pool.ntp.org -p 2.openwrt.pool.ntp.org -p 3.openw
 8833  2318 root     S     1120   0%   0% udhcpc -p /var/run/udhcpc-eth0.pid -s /lib/netifd/dhcp.script -f -t 0 -i eth0 -x hostname:OpenWrt -C -R -O 121
 1943     1 logd     S      992   0%   0% /sbin/logd -S 64
 1615     1 ubus     S      972   0%   0% /sbin/ubusd
 2202     1 root     S      936   0%   0% /usr/sbin/dropbear -F -P /var/run/dropbear.1.pid -p 22 -K 300 -T 3
 1648     1 root     S      812   0%   0% /sbin/urngd
 1616     1 root     S      748   0%   0% /sbin/askfirst /usr/libexec/login.sh
 1617     1 root     S      748   0%   0% /sbin/askfirst /usr/libexec/login.sh
   10     2 root     SW       0   0%   0% [ksoftirqd/0]
   11     2 root     IW       0   0%   0% [rcu_sched]
   25     2 root     SW       0   0%   0% [migration/3]
   20     2 root     SW       0   0%   0% [migration/2]
   15     2 root     SW       0   0%   0% [migration/1]
  388     2 root     SW       0   0%   0% [kcompactd0]
 5246     2 root     IW       0   0%   0% [kworker/u8:0-ev]
 5244     2 root     IW       0   0%   0% [kworker/3:1-eve]
 7059     2 root     IW       0   0%   0% [kworker/1:2-eve]
   26     2 root     SW       0   0%   0% [ksoftirqd/3]
   16     2 root     SW       0   0%   0% [ksoftirqd/1]
 5565     2 root     IW       0   0%   0% [kworker/1:0-rcu]
 1166     2 root     SW       0   0%   0% [usb-storage]
 8373     2 root     IW       0   0%   0% [kworker/0:2-eve]
 7826     2 root     IW       0   0%   0% [kworker/u8:2-ev]
 7159     2 root     IW       0   0%   0% [kworker/0:0-eve]
   21     2 root     SW       0   0%   0% [ksoftirqd/2]
 1017     2 root     SW       0   0%   0% [scsi_eh_0]
   12     2 root     SW       0   0%   0% [migration/0]
  822     2 root     IW<      0   0%   0% [kthrotld]
  784     2 root     IW<      0   0%   0% [kworker/0:1H-kb]
  688     2 root     SW       0   0%   0% [watchdogd]
  820     2 root     SW       0   0%   0% [kswapd0]
  857     2 root     IW<      0   0%   0% [acpi_thermal_pm]

Intel i350 is not available - i think it's because of missing drivers.

Looks like i350 requires the igb driver which isn't currently built.

Re the Realtek NICs:

2) Given you can run the system from a USB drive, could you test with an OpenWrt 22.03.3 image to see whether you see the same performance with the Realtek NICs as WAN and LAN as you have them in Gargoyle? Noting that OpenWrt 22.03.3 should see your i350 as it includes the igb driver, it would be useful to see performance numbers for mixed combinations too, i.e. i350 as WAN and Realtek as LAN and vice versa (and if your i350 is a multiport variant, i350 as both WAN and LAN). Monitoring CPU stats with top while doing this would also be helpful.

So i tested Openwrt on my machine:
1. On intel i350 set as lan and wan it works at full speed.
2. Problems appear when i set wan to one of realtek nic and lan to intel i350 then it works only at 25-30 mb.
3. When i set wan to one of port of intel i350 and lan to rest of ports of intel and realteks then download speed on rlt is almost full.

This shows that problem is when realtek is set as a wam port.
Why?

[pythonic]

I don't have answers for you sorry.

13% sirq as shown in your top output means you aren't CPU limited.

You might also want to look at whether irq distribution is clustered onto one core - if so re-assigning NIC irqs to other cores might help.

With the realtek NICs it may be a result of them sharing an irq and maybe other driver resources producing non-optimal network stack behaviour - they're generally only used in desktops with a single network connection and don't cause any noticeable issues when solo (i.e. only one realtek NIC managed by the driver).

There's a reason that many people prefer intel NICs for heavy duty applications (servers, routers etc) because the silicon and the firmware/drivers are superior and designed to be among the best performers; realtek stuff on the other hand is generally designed and built to a price-point rather than a performance standard, and their handling of Linux driver code across numerous hardware variations leaves a lot to be desired.

BTW: the intel NIC drivers have been added to Gargoyle's 1.15.x x86_64 config and should be included in any new builds released.

[pythonic]

So i tested Openwrt on my machine:
1. On intel i350 set as lan and wan it works at full speed.
2. Problems appear when i set wan to one of realtek nic and lan to intel i350 then it works only at 25-30 mb.
3. When i set wan to one of port of intel i350 and lan to rest of ports of intel and realteks then download speed on rlt is almost full.

Additional thought: given that the problem is visible in OpenWrt, you will likely get much more help in the OpenWrt forums. If you can get the problem resolved in OpenWrt, any fix is likely to then be implementable in Gargoyle.

[tob99]

Additional thought: given that the problem is visible in OpenWrt, you will likely get much more help in the OpenWrt forums. If you can get the problem resolved in OpenWrt, any fix is likely to then be implementable in Gargoyle.

That would be right. I think that there is a problem with rtl driver, maybe i need some special one for my rtl nics.

I downloaded Lantis image from February 19th and as you've mentioned it has intel i350 driver included. It solves my problems. Now i use my dell terminal with that image burned on ssd drive and it works without any issues for almost a week.
Thank you once again for your help Pythonic.

[Maniac]

@Lantis any chance you could add the Belkin F9K1115v2 to the ath79 builds? It's supported in OpenWRT. Thanks!

[ispyisail]

Flashed my C7