Experiences building cheap x86-based router

General discussion about Gargoyle, OpenWrt or anything else even remotely related to the project

Moderator: Moderators

Post Reply
professorpip
Posts: 2
Joined: Thu Jan 21, 2021 5:54 am

Experiences building cheap x86-based router

Post by professorpip »

I've been using Gargoyle for years at home. I have one Buffalo WZR-HP-G300NH. I also have two TP-Link TL-WDR4300 v1's. It has been excellent. Thank you to all the creators. :)

Internet to our house is via Cable (Virgin Media in the UK) - and have recently upgraded to a 500Mbps package.

Due to Covid lockdown the family is working + studying from home - so we rely on good internet connections. One of us is also a keen gamer.

Originally - the Buffalo was the router and an wifi AP. The Virgin box was in dumb "cable modem" mode. The WD4300's were just APs around the house to expand our wifi range.

A few months ago I noticed my work VPN connection was not working smoothly. It coincided when our son was PC gaming downstairs (Ark Survival seemed to really stretch the network!). It turned out that the Buffalo G300NH was over-stretched with all the increased bandwidth usage.

I swapped the WD4300 over to be the main router & AP (as it is a bit faster than the Buffalo) - but it still couldn't sustain much more than 150-200Mpbs throughput in a internet speedtest (wired, not over wireless)

I decided to try out an x86 router - so I got a HP 610 plus from ebay (a thin client with an dual core AMD T56N x86-64 cpu). It has a pcie slot, so I put an old Intel dual e1000 gigabit nic into it. I did know the the T620 plus has a better cpu (with AES acceleration) and so is what many people use for diy x86 routers, but I couldn't get one at a reasonable price - and the 610 was very cheap at the time.

At first I tried pfsense (which is what most people seem to run on DIY x86 routers). It was fine, but I preferred the UI of opnsense (which shares the same Freebsd foundation as pfsense). It worked ok - not as easy to get to grips with as Gargoyle is, but certainly powerful and flexible (and can see why people use it sometimes in professional setups).

I noticed I was not getting the full 500Mbps speed through the router. It was peaking at around 250Mbps. It seems that the T56N APU is just not that powerful.

I realised I really missed the familiar Gargoyle interface, and I wondered if Gargoyle (being Openwrt / Linux based) might perform better than the pfsense/opnsense (which is Freebsd based). In particular - I read that openwrt can use multiple cpu cores for routing, whereas pfsense only uses one cpu core for routing.

So I followed instructions on this forum, and built the latest 1.13 version of Gargole for x86-64 from source from git (Using a Ubuntu 20.04 distro on a virtual machine). It took about 12-16 hours to build - but I got the image to write to a usb stick (using etcher), and it booted fine on the T610 plus. I did expand the ext4 filesystem (via parted / resizefs) - so that I can add all the extra packages I want via opkg (which is very useful)

I was back with Gargoyle on the T610 plus router!

In terms of performance so far - I did have one issue - my very old Intel e1000 NIC suffered from from a hang error whenever I did a speedtest. This issue is covered in various places - if you google "e1000e-reset-adapter-unexpectedly-detected-hardware-unit-hang" - you'll find lots of info.

The fix that worked for me was to install ethtool via opkg and set

Code: Select all

ethtool -K eth0 tso off
to turn off the TCP Segmentation Offload feature on the Intel nic. It does increase the cpu load a bit as result. (I never saw this issue when running pfsense/opnsense). However - after this the box was & is running stably.

The upshot of this is I now see speedtests of up to 420Mps - the best so far (and Gargoyle is faster in my particular use case with this limited cpu than opnsense was) . However it's still short of the maximum ISP package speed of 520Mbps (this is not theoretical - I've actually measured this direct from the cable modem so that I know what the internet speed to the house can actually reach).

So my next step is to try an upgraded x86 box. I also have a nice HP T820 thin client also bought cheaply on ebay (I got mine for 70GBP shipped). It has a Pentium G3220 3Ghz cpu, 4GB ram, and a 16GB SSD (which can be upgraded). It's typically not considered as a x86 DIY router candidate as it has only one onboard NIC (Intel I217LM) and no PCIE slot. At that price though, with an SSD upgrade, it becomes a very nice silent desktop Linux or Win10 box (which is what I've been using it as until now). The key discovery for me was it does have an internal spare mini pcie slot - originally intended for use as a factory-fitted HP Mini PCIe 100 Mbps SC Fiber NIC option. I have ordered a cheap mini PCIE gigabit network card from ebay - (I went for a "syba" brand sold on ebay and amazon) There different brands - some less than 10USD delivered from china. The cheap ones have Realtek RTL8111F chipsets (should work with the r8169 driver). For about 30-40 USD I have seen similar Intel chipset NICs which may perform better. And there are some versions with dual gigabit ports. I will take some photos of how I'm mounting the Lan socket at the back of the T820's chassis - but it is very neat, with no dangling wires etc. Obviously if you use the mini-pcie slot for a wired NIC, you can't put another wifi card in. It does have an iwl6000 wifi chipset built into the motherboard, but I don't think that supports access point mode...(but I will stick to my plan of having separate AP's anyway).

So - in conclusion - I reckon a used HP T820 with a cheap mini PCIe 2nd NIC is a path to a sub-100USD x86 DIY router
that I expect can easily cope with Gigabit speeds
(and can happily run any of Gargole, Openwrt, pfsense, or opnsense). I liked opnsense, but for my home use-case Gargole is just perfect! ;) I will post updates in a few days with the kind of performance I can get. But I thought this was quite an interesting journey - hence wanted to share with this forum, and would be happy to answer any questions if others like me are considering such a diy project.

ispyisail
Moderator
Posts: 5180
Joined: Mon Apr 06, 2009 3:15 am
Location: New Zealand

Re: Experiences building cheap x86-based router

Post by ispyisail »

+1

fifonik
Posts: 124
Joined: Fri Dec 02, 2016 3:52 am
Location: Brisbane, AU

Re: Experiences building cheap x86-based router

Post by fifonik »

Thanks for sharing this.

Could you advise, what the speeds mentioned with Quota/QoS/BW Monitoring or without?

professorpip
Posts: 2
Joined: Thu Jan 21, 2021 5:54 am

Re: Experiences building cheap x86-based router

Post by professorpip »

I don't have much setup in terms of Quota's or QoS - so I can't say.

What I can say is that the HP610 Plus was definitely cpu-bottlenecked, and my new HP 810 (with at 3 Ghz Pentium G3220) is definitely not.

Before I was getting 350-400 Mbs max throughput, with the cpu maxed out. With the new box - I can reach the max speed of my internet service (540Mpbs), with room to spare, as the cpu doesn't go above 35% on one core (and is basically nothing on the other). I think therefore it could easily cope with gigabit speeds.

I am still getting the e1000e hang issue - that's twice now on 2 different Intel NICs on 2 different PCs. I can fix it by disabling the checksum offloading... I never saw this issue with pfsense of opnsense, so I wonder if it is a linux kernel driver issue. Anyway - it is easily resolved via the ethtool command I mentioned in my first post -albeit at the expense of slightly higher cpu usage.

Now I have my router, and all 3 access points on 1.13git. I am enjoying using the fast wifi-handoffs feature between APs - so am getting un-interrupted wifi across all floors & areas of the house.

stuartbh
Posts: 76
Joined: Sun Oct 20, 2019 4:18 pm

Re: Experiences building cheap x86-based router

Post by stuartbh »

Professorpip,

Do you happen to have a link to the instructions that worked for building Gargoyle for x86? I am interested to build it not for a USB but to run in a VM to test and eventually via USB.

Thanks!
Very Respectfully,

Stuart, N3GWG
Computer Scientist

pythonic
Posts: 230
Joined: Mon Mar 11, 2019 5:47 am
Location: Australia

Re: Experiences building cheap x86-based router

Post by pythonic »

re: building, see the developer documentation.

@ispysail also makes builds available from time to time - the most recent at this time seems to be linked here.

Post Reply