x86 and x86_64 woes

Discuss the technical details of Gargoyle and ongoing development

Moderator: Moderators

Post Reply
robnitro
Posts: 84
Joined: Sat Feb 26, 2011 4:50 pm

x86 and x86_64 woes

Post by robnitro »

I have been working on setting up an x86 based router as I want qos that can handle more than 50 mbit up/down. The commercial routers have at most arm 1.xx ghz cpu's which aren't even comparable to a cheapo low end x86 cpu.

But as I have found, building is a hassle- especially when you want to do x86_64 (to get proper SMP support). Despite the weirdness in the gargoyle build environment - as it uses openwrt build env as a base- I finally got the packages I need to work out.

However, I hit another frustration which occurs whether I use my x86_64 or 32 bit builds or other premade gargoyle x86 builds:

If traffic is going on up, down, or both- changing anything in the qos down or up page will crash the router. There's no message because it becomes either a hard freeze or reboots automatically (depending on kernel options). With little or no traffic, there is no crash and qos updates fine.
Even maxing out the qos at 300/300mbit (yes, wow and on a dual core 1.2 ghz core 2!) the cpu load is below 50% total!

VM's were notoriously slow, whether using vmxnet, virtio, or e1000e drivers. I have tried all the NIC tweaks on the host and guest os.
I have the same issue with pfsense- and I think it's because VMs add latency to the network stack which throws off the queuing capability of both systems.

Any idea what could be causing this or how I could get an actual trace that could help me find out? I suppose I could try to sacrifice a 120Gb sata drive to load an OS that needs less than 100mb to eliminate the usb flash issue (I have used different ports and flash drives).

robnitro
Posts: 84
Joined: Sat Feb 26, 2011 4:50 pm

Re: x86 and x86_64 woes

Post by robnitro »

Same thing with using sata/ide as boot.

Doing moderate load, 233 up 220 mbit down. Cpus show 12% and 20% according to htop.

Edited speed by a tiny bit higher- 220,000 to 230,000 and after 10 seconds console /ssh/ web unresponsive along with the bandwith going to zero.

Have to hard reset the pc.
No issues under this or higher loads on pfsense, so I don't think it's a hardware or nic issue!

robnitro
Posts: 84
Joined: Sat Feb 26, 2011 4:50 pm

Re: x86 and x86_64 woes

Post by robnitro »

For now I am going to use pfsense. If theres a bug fix, I will try again. I have extra usb sticks to DD and run if I hear anything. I spent too many hours compiling different packages and kernel settings to get it to work- only to see that even the bare bones single cpu version did the same, doh!

Even though many complained about the qos, I found it to be very similar as gargoyle's , despite not having fq_codel. The hsfc is close enough for me to learn what to do, and their classifications aren't as simple as gargoyle, where I could prioritize small packets or easily change by ports without having to edit floating firewall rules.

tapper
Moderator
Posts: 1076
Joined: Sun Oct 13, 2013 5:49 pm
Location: Stoke-on-trent UK

Re: x86 and x86_64 woes

Post by tapper »

robnitro wrote:For now I am going to use pfsense. If theres a bug fix, I will try again. I have extra usb sticks to DD and run if I hear anything. I spent too many hours compiling different packages and kernel settings to get it to work- only to see that even the bare bones single cpu version did the same, doh!

Even though many complained about the qos, I found it to be very similar as gargoyle's , despite not having fq_codel. The hsfc is close enough for me to learn what to do, and their classifications aren't as simple as gargoyle, where I could prioritize small packets or easily change by ports without having to edit floating firewall rules.
Hi opensense will have fq_codel. in a update.
I read it on a email list but cant find the email rite now but it said it will not be long. The guys from opensense are doing some good work but it's a bit much for my needs.
see here https://opnsense.org/opnsense-16-1-12-released/
Linksys WRT3200ACM
NETGEAR Nighthawk R7800
NETGEAR R6260

robnitro
Posts: 84
Joined: Sat Feb 26, 2011 4:50 pm

Re: x86 and x86_64 woes

Post by robnitro »

Thanks for that update.
I think fq_codel might not actually help us though. With HSFC on pfsense, like gargoyle, the queues themselves handle the splitting.

Fq_codel from what I read seems to be more for an easy knobless setup.
After learning how to tweak hsfc, I'm spoiled with that :)

The only confusion in pfsense is where to put codel on. I put it on every class, but does it also go on the interface and main leaf (eg WAN/LAN and qInternet)?

Post Reply