x86 and x86_64 woes

[robnitro]

I have been working on setting up an x86 based router as I want qos that can handle more than 50 mbit up/down. The commercial routers have at most arm 1.xx ghz cpu's which aren't even comparable to a cheapo low end x86 cpu.

But as I have found, building is a hassle- especially when you want to do x86_64 (to get proper SMP support). Despite the weirdness in the gargoyle build environment - as it uses openwrt build env as a base- I finally got the packages I need to work out.

However, I hit another frustration which occurs whether I use my x86_64 or 32 bit builds or other premade gargoyle x86 builds:

If traffic is going on up, down, or both- changing anything in the qos down or up page will crash the router. There's no message because it becomes either a hard freeze or reboots automatically (depending on kernel options). With little or no traffic, there is no crash and qos updates fine.
Even maxing out the qos at 300/300mbit (yes, wow and on a dual core 1.2 ghz core 2!) the cpu load is below 50% total!

VM's were notoriously slow, whether using vmxnet, virtio, or e1000e drivers. I have tried all the NIC tweaks on the host and guest os.
I have the same issue with pfsense- and I think it's because VMs add latency to the network stack which throws off the queuing capability of both systems.

Any idea what could be causing this or how I could get an actual trace that could help me find out? I suppose I could try to sacrifice a 120Gb sata drive to load an OS that needs less than 100mb to eliminate the usb flash issue (I have used different ports and flash drives).

[robnitro]

Same thing with using sata/ide as boot.

Doing moderate load, 233 up 220 mbit down. Cpus show 12% and 20% according to htop.

Edited speed by a tiny bit higher- 220,000 to 230,000 and after 10 seconds console /ssh/ web unresponsive along with the bandwith going to zero.

Have to hard reset the pc.
No issues under this or higher loads on pfsense, so I don't think it's a hardware or nic issue!

[robnitro]

For now I am going to use pfsense. If theres a bug fix, I will try again. I have extra usb sticks to DD and run if I hear anything. I spent too many hours compiling different packages and kernel settings to get it to work- only to see that even the bare bones single cpu version did the same, doh!

Even though many complained about the qos, I found it to be very similar as gargoyle's , despite not having fq_codel. The hsfc is close enough for me to learn what to do, and their classifications aren't as simple as gargoyle, where I could prioritize small packets or easily change by ports without having to edit floating firewall rules.

[tapper]

For now I am going to use pfsense. If theres a bug fix, I will try again. I have extra usb sticks to DD and run if I hear anything. I spent too many hours compiling different packages and kernel settings to get it to work- only to see that even the bare bones single cpu version did the same, doh!

Even though many complained about the qos, I found it to be very similar as gargoyle's , despite not having fq_codel. The hsfc is close enough for me to learn what to do, and their classifications aren't as simple as gargoyle, where I could prioritize small packets or easily change by ports without having to edit floating firewall rules.

Hi opensense will have fq_codel. in a update.
I read it on a email list but cant find the email rite now but it said it will not be long. The guys from opensense are doing some good work but it's a bit much for my needs.
see here https://opnsense.org/opnsense-16-1-12-released/

[robnitro]

Thanks for that update.
I think fq_codel might not actually help us though. With HSFC on pfsense, like gargoyle, the queues themselves handle the splitting.

Fq_codel from what I read seems to be more for an easy knobless setup.
After learning how to tweak hsfc, I'm spoiled with that

The only confusion in pfsense is where to put codel on. I put it on every class, but does it also go on the interface and main leaf (eg WAN/LAN and qInternet)?