Setting up a Raspberry Pi with 2 Network Interfaces as a simple router(jeffgeerling.com)
jeffgeerling.com
Setting up a Raspberry Pi with 2 Network Interfaces as a simple router
https://www.jeffgeerling.com/blog/2021/setting-raspberry-pi-2-network-interfaces-very-simple-router
45 comments
Old but relevant, I have used an espressobin as a router for years and the performance I am getting is similar to much more expensive semi-professional routers. https://blog.tjll.net/building-my-perfect-router/
For those who wish to undergo lesser trouble to get similar performance/security there are number of Single Board Computers(SBCs) with OpenWrt support[1] sometimes with official support like those from Friendly Elec.
[1]https://openwrt.org/toh/views/toh_single-board-computers
[2]https://www.friendlyarm.com/index.php?route=product/category...
[1]https://openwrt.org/toh/views/toh_single-board-computers
[2]https://www.friendlyarm.com/index.php?route=product/category...
espressobin is currently #90 on the openwrt table of hardware for single board computers you linked.
personally I'd way rather have a system that boots a good Debian rather than openwrt, & the espressobin will fair many times better/be much easier than many of these openwrt-centric alternatives. and it will happily run openwrt too. this line of chips from Marvell has done a good job over the years keeping good upstream/mainline kernel & u-boot support. whereas openwrt often good es out of itcs way with out of tree patches to support all kinds of difficult hardware environments.
personally I'd way rather have a system that boots a good Debian rather than openwrt, & the espressobin will fair many times better/be much easier than many of these openwrt-centric alternatives. and it will happily run openwrt too. this line of chips from Marvell has done a good job over the years keeping good upstream/mainline kernel & u-boot support. whereas openwrt often good es out of itcs way with out of tree patches to support all kinds of difficult hardware environments.
Number of SBCs listed there boot debian as well.
My suggestion wasn't a debian vs OpenWrt one, But rather a 'If you like OpenWrt, then you got SBC options too'.
My suggestion wasn't a debian vs OpenWrt one, But rather a 'If you like OpenWrt, then you got SBC options too'.
I'm constantly looking for good priced SBC with cellular connectivity - that chart shows only 2/3 with modems and all appear to be via external modules.
Anyone know of any?
Anyone know of any?
> This page shows devices which have a LTE modem built in and are supported by OpenWrt.
https://openwrt.org/toh/views/toh_lte_modem_supported
It looks like this table is neither current nor complete though. And there's a different table of OpenWRT compatible devices that have a battery as well.
> [The Amarok (GL-X1200) Industrial IoT Gateway has] 2x SIM card slots for 2x 4G LTE modems (probably miniPCI-E so maybe upgradeable to 5G later), external antenna connectors for the LTE modems, MicroSD, #OpenWRT: https://store.gl-inet.com/collections/4g-smart-router/produc...
The Turris Omnia also has 4G LTE SIM card support (and LXC in their OpenWRT build). https://openwrt.org/toh/turris/turris_omnia
There's also a [Dockerized] x86 build of OpenWRT that probably also supports Mini PCI-E modules for 4G LTE, LoRa, and 5G. Route metrics determine which [gateway] route is tried first.
From "How much total throughput can your wi-fi router really provide?" https://news.ycombinator.com/item?id=26596395 :
> In 2021, most routers - even with OpenWRT and hardware-offloading - cannot actually push 1 Gigabit over wired Ethernet, though the port spec does say 1000 Mbps
https://openwrt.org/toh/views/toh_lte_modem_supported
It looks like this table is neither current nor complete though. And there's a different table of OpenWRT compatible devices that have a battery as well.
> [The Amarok (GL-X1200) Industrial IoT Gateway has] 2x SIM card slots for 2x 4G LTE modems (probably miniPCI-E so maybe upgradeable to 5G later), external antenna connectors for the LTE modems, MicroSD, #OpenWRT: https://store.gl-inet.com/collections/4g-smart-router/produc...
The Turris Omnia also has 4G LTE SIM card support (and LXC in their OpenWRT build). https://openwrt.org/toh/turris/turris_omnia
There's also a [Dockerized] x86 build of OpenWRT that probably also supports Mini PCI-E modules for 4G LTE, LoRa, and 5G. Route metrics determine which [gateway] route is tried first.
From "How much total throughput can your wi-fi router really provide?" https://news.ycombinator.com/item?id=26596395 :
> In 2021, most routers - even with OpenWRT and hardware-offloading - cannot actually push 1 Gigabit over wired Ethernet, though the port spec does say 1000 Mbps
Thanks for this. I've been keeping an eye out for home firewall hardware that won't break the bank, spike my power bill, and hopefully still keep up with most of my theoretically-1-Gbps home connection.
Not sure if you're looking to use pfSense, but if you are, and you want to get gigabit for a single TCP connection, choose carefully among the SBC options. See this for more detail:
https://teklager.se/en/knowledge-base/apu2-1-gigabit-through...
For me, I've discovered the excellent Supermicro X10SLM+-F motherboard. I've bought them off eBay for as little as $145 with 32 gigs of ECC RAM and a 4-core Xeon E3-1220v3 included. Every one I've grabbed runs flawlessly. That board uses around 25w at idle with the BMC running, and can host more than just a pfSense VM. You still have to supply a case and a high quality PSU (don't skimp here!), but those don't have to break the bank.
(If you want to run your router in a VM and pass the LAN controllers through to it, I can recommend the X10SLM+-LN4F, which adds two more gigabit ports. Pricing should be similar.)
If I had to do this on a tighter budget, I have tried OpenWRT on an Rpi4 and it was definitely workable. Didn't have a few features I was looking for, and I was already running a Supermicro anyway for other workloads, so I went back to using a pfSense VM, but YMMV.
https://teklager.se/en/knowledge-base/apu2-1-gigabit-through...
For me, I've discovered the excellent Supermicro X10SLM+-F motherboard. I've bought them off eBay for as little as $145 with 32 gigs of ECC RAM and a 4-core Xeon E3-1220v3 included. Every one I've grabbed runs flawlessly. That board uses around 25w at idle with the BMC running, and can host more than just a pfSense VM. You still have to supply a case and a high quality PSU (don't skimp here!), but those don't have to break the bank.
(If you want to run your router in a VM and pass the LAN controllers through to it, I can recommend the X10SLM+-LN4F, which adds two more gigabit ports. Pricing should be similar.)
If I had to do this on a tighter budget, I have tried OpenWRT on an Rpi4 and it was definitely workable. Didn't have a few features I was looking for, and I was already running a Supermicro anyway for other workloads, so I went back to using a pfSense VM, but YMMV.
Check also the PCEngines APU boards at https://pcengines.ch
Also interesting are the NanoPi R2S and R4S at https://www.friendlyarm.com
Especially those by PCEngines are designed for that job; I've used their Wrap boards in the past as firewalls with PfSense and they were rock solid.
No affiliation with any of them, just a satisfied customer.
Also interesting are the NanoPi R2S and R4S at https://www.friendlyarm.com
Especially those by PCEngines are designed for that job; I've used their Wrap boards in the past as firewalls with PfSense and they were rock solid.
No affiliation with any of them, just a satisfied customer.
Can confirm, I have been using pfSense on one apu board, and debian on another older 2c13 board. they are such nifty devices with a very simple case.
Big fan.
Big fan.
I use a pcengines apu2. It's x64 and runs ubuntu. 3 onboard gbe ports. Passively cooled/silent, boots from SD.
Do you run OpenVPN on it? My biggest issue with consumer routers and SBC routers is their inability to decrypt 100+ Mbps of AES-256 data.
I thought I'd post this since other people might find it useful—especially for quick testing purposes, as you can get a little hardware router up and running pretty quickly, and so far in my testing the Pi 4 model B can put through at least 750-800 Mbps with no sweat, and the Compute Module 4 can do 940+ Mbps assuming you use a PCI Express NIC instead of using a USB 3.0-to-Gigabit NIC.
I had to do this as part of my testing of two new Compute Module 4-based router boards, both of which may make some decent low-cost and tiny Gigabit routers / firewalls / VPN appliances, depending on your needs.
I had to do this as part of my testing of two new Compute Module 4-based router boards, both of which may make some decent low-cost and tiny Gigabit routers / firewalls / VPN appliances, depending on your needs.
For about 10 months my home router has been a Pi4 running Ubuntu 20.04. I use a realtek based USB 3.0 to gigabit Ethernet adapter. With the Pi4 overclocked to 2Ghz I get around 930Mbs. The router is located inside a distribution panel in an unconditioned garage. It has been rock solid with only a $10 generic case with fan ordered off eBay to protect it.
I use a similar method to share Internet with my Tandy 1000 via an old laptop I've used as a server for years. LAN via Ethernet coming in, network forwarding via iptables, and serial null modem cable to the Tandy. Unfortunately, I don't get anywhere near 750-800 Mbps but 6 Kbps :)
You should take a look at serial-to-ethernet converters like the USR-TCP232-T2. I use one with my VIC-20 to telnet into BBSs. Uses a modified AT command set, so you can actually control it a bit like a modem. Picked it up for $10 off eBay a few years back.
Hey, thanks! I might actually use this for another project.
Super interested in what boards you are using and/or if they're custom! Would love to have a Pi 4 based home router with the 2nd NIC on the PCIe bus.
Jeff has published much about his Pi adventures. Some examples:
https://pipci.jeffgeerling.com/
https://github.com/geerlingguy
Oh holy cow. It's that guy ha. Will dig deeper - thanks for the links!
The two I'm currently testing are Seeed Studio's router board and DFRobot's as well.
Just got the order in for the Seeed Studio board. I'm sure I have a good wait time with shipping but looking forward to replacing some Ubiquiti gear after their drama.
Thanks for the reply as it totally inspired me to build my own router like it's 2003 =)
Thanks for the reply as it totally inspired me to build my own router like it's 2003 =)
Hi Jeff
Did you get 750-800 out of the box with a Pi4 and USB ethernet? Asking because mine was around 300 and needed some manual tweaking (see also https://news.ycombinator.com/item?id=27231705)
Did you get 750-800 out of the box with a Pi4 and USB ethernet? Asking because mine was around 300 and needed some manual tweaking (see also https://news.ycombinator.com/item?id=27231705)
Jeff is showcasing his compute boards again, but for everyone else the stock Rpi4 with a cheapo USB3 Ethernet adapter works just great and can almost fully saturate 1Gbps - good enough for the majority of home users.
Here's an article on how to install OpenWRT on that (not that you can't just use any distro really, slap ntopng on top for stats and Jeff's your uncle).
https://www.zahradnik.io/raspberry-pi-as-a-home-router
Here's an article on how to install OpenWRT on that (not that you can't just use any distro really, slap ntopng on top for stats and Jeff's your uncle).
https://www.zahradnik.io/raspberry-pi-as-a-home-router
An alternative to worrying about two ethernet ports on the Pi itself is getting a managed switch with port tagging, and just setting up multiple VLANs on the Pi itself. I'm doing that myself with another small single-port board, works quite nicely.
The effort of learning how to set up and maintain vlans is WELL worth it.
for example, many of my machines are on a private vlan that cannot get out (except through a proxy). This is one of the most effective ways of managing your devices since this means by default you will have to opt-in to talk to the internet.
for example, many of my machines are on a private vlan that cannot get out (except through a proxy). This is one of the most effective ways of managing your devices since this means by default you will have to opt-in to talk to the internet.
Indeed that's another way of doing it, quite elegant, too.
For those who get poor performance out of this, I recommend https://www.kernel.org/doc/html/latest/networking/scaling.ht...
I had 300-400Mbit/s out of the box and got it up to ~900 with the right set up.
I had 300-400Mbit/s out of the box and got it up to ~900 with the right set up.
There are a lot devices in the range of Raspberry Pi that is commercially made for routers with power-supply and enclosure and FCC certified running Openwrt.
I wish all SBCs can have two network interfaces, then wifi/bluetooth, a RTC and a Watchdog and eMMC and FCC and housing. CSI and DSI and HDMI etc can be optional.
I wish all SBCs can have two network interfaces, then wifi/bluetooth, a RTC and a Watchdog and eMMC and FCC and housing. CSI and DSI and HDMI etc can be optional.
Has anyone found a way to enable receive side scaling on Raspberry Pi 4?
I'm working on a project recently and the main bottleneck I'm seeing is that all eth0 interrupts go to a single core, and that core gets swamped with interrupts until it can't handle them anymore and ksoftirqd maxes out at 100% usage on a single core.
This seriously hurts the throughput. I know with RSS I would be able to distribute the eth0 hardware interrupts into 4 queues, with each queue being balanced by irqbalance onto it's own core.
However, in all my searching it appears that the Raspberry Pi hardware/driver is not capable of receive side scaling, and can only put eth0 interrupts into a single queue.
I would love to be told otherwise, if anyone else has found a workaround for this.
I'm working on a project recently and the main bottleneck I'm seeing is that all eth0 interrupts go to a single core, and that core gets swamped with interrupts until it can't handle them anymore and ksoftirqd maxes out at 100% usage on a single core.
This seriously hurts the throughput. I know with RSS I would be able to distribute the eth0 hardware interrupts into 4 queues, with each queue being balanced by irqbalance onto it's own core.
However, in all my searching it appears that the Raspberry Pi hardware/driver is not capable of receive side scaling, and can only put eth0 interrupts into a single queue.
I would love to be told otherwise, if anyone else has found a workaround for this.
The NanoPi R4S has a RTL8169 controller with MSI-X capabilities. Post irqbalance setup I verified it's properly distributing IRQs. I'm using mine as a transparent bridge and network monitor. https://news.ycombinator.com/item?id=27236081
Capabilities: [b0] MSI-X: Enable+ Count=4 Masked-Interesting, I'll have to check that device out. I'm using Raspberry Pi 4b and it does not have the right capabilities, ethtool just says the operation is not supported:
sudo ethtool -x eth0
Cannot get RX ring count: Operation not supported
Back in 2015, I ran a FreeBSD router on my Rasberry Pi 2B. Instead of relying on two ports, I divided a single port using VLANs, and connected my modem to a L2 managed switch. This configuration is colloquially known as "router on a stick." Performance wasn't amazing, but neither was my rual ISP, so I ran with it for a year. I eventually replaced it with a Beaglebone setup in the same configuration, but with OpenBSD, and an IPSec VPN. That setup remained until I moved out a couple years later. I have been wanting to try it again with a PoE-capable RPi - removing a dedicated power cable and only requiring a single cable would truly make it a "router on a stick."
I did this for a hacker space workshop many years ago where I converted a raspi into a wifi router that performed MITM attacks.
So I had all the participants in the workshop connect to this free wifi network at the beginning, because it was available and it was free internet while you were there.
Then at some point during the workshop I had them all visit like facebook without https (you could still do that then) and they were greeted by a poem instead.
This was a workshop on how to protect yourself in daily online life. So the basic tricks like use https, check your browser, bla bla.
So I had all the participants in the workshop connect to this free wifi network at the beginning, because it was available and it was free internet while you were there.
Then at some point during the workshop I had them all visit like facebook without https (you could still do that then) and they were greeted by a poem instead.
This was a workshop on how to protect yourself in daily online life. So the basic tricks like use https, check your browser, bla bla.
Been using a rpi zero as a pihole for over a year now. Yay for ads, I dont see them. Only 70-90% of all requests are blocked, not much and I have like 10-15 devices on at all times so it works.
I similarly documented my pi network setup. NAT/DHCP/DNS: https://gist.github.com/cellularmitosis/8294f0800e6d4b402277... WiFi AP: https://gist.github.com/cellularmitosis/d1b87d9834ef53c54695...
The problem I've found using the RPI is that it doesn't have the CPU instructions for fast AES, so if you want to route your traffic through a VPN you're stuck with 40-50Mbps.
Now with VRF-lite in the kernel, it's super simple to route all traffic through a VPN, just put your WAN iface in one VRF and the OVPN/WG interface and your LAN interface in another one, masquerade outgoing traffic on the VPN interface and call it a day. Ofc you need to connect to a VPN provider you trust. TorrentFreak might be a good resource to find reliable ones (the ones that protect ThePirateBay for example).
I'm probably just gonna buy a teeny x86 board, ARM for computers is still weird and incompatible here and there (why is there a raspian when there's already debian, why is there a Manjaro version per popular ARM device while there's only one for x86_64 and one for x86 on the CISC side).
Now with VRF-lite in the kernel, it's super simple to route all traffic through a VPN, just put your WAN iface in one VRF and the OVPN/WG interface and your LAN interface in another one, masquerade outgoing traffic on the VPN interface and call it a day. Ofc you need to connect to a VPN provider you trust. TorrentFreak might be a good resource to find reliable ones (the ones that protect ThePirateBay for example).
I'm probably just gonna buy a teeny x86 board, ARM for computers is still weird and incompatible here and there (why is there a raspian when there's already debian, why is there a Manjaro version per popular ARM device while there's only one for x86_64 and one for x86 on the CISC side).
What daughterboard is he using here? It would be useful for me to carry around a little pi gateway with more than 1 ethernet.
Dual Gigabit Board:
https://www.seeedstudio.com/Rapberry-Pi-CM4-Dual-GbE-Carrier...
https://www.seeedstudio.com/Rapberry-Pi-CM4-Dual-GbE-Carrier...
Someone already answered, but I'm actually producing a few boards that work with either the Raspberry Pi or Jetson Nano/NX with dual ethernet via the native Ethernet and (miniPCIE or USB) respectively. Will open source board designs + sell them for near cost probably, had to order 5 to wave the engineering fee. I believe the engineering firm is the same as the other posted board haha.
I've found that people dismiss the idea of a Pi as a router, but in practice it actually works really well. Performance is really close to 1Gb and the software stack is as versatile and maintainable as it can be.
I didn't dismiss it because of performance. I dismissed it due to not being able to run something like pfsense or opnsense. I know it's possible to do everything with Linux and no shiny web frontend, but I don't really want to. If everything was IPv6 it would be easier but I'm a bit scared of getting stuff like NAT rules to work and the router is way too important to be a toy (my household depends on it for work).
[deleted]
wholeot of nonsense you don't have to learn if you just use systemd. dhcpd? bah. systemd bundles a very adequate far better integrated DHCP server in systemd-networkd. all it takes is:
DHCPSsrver=yes
in the eth1.network file. Address=10.99.98.1/24 to pick the pool. IPMasquerade=yes will make it forward too. Done. In a standard, understandable, uniform way. Unlike picking & choosing some random daemons to serve random jobs, unlike hacking up some random iptables script you've duck taped together.
I love love love that we're making videos to show people the basics of compute. Adore it. But wow this is souch less ad-hoc & souch more orderly & easier with systemd.
DHCPSsrver=yes
in the eth1.network file. Address=10.99.98.1/24 to pick the pool. IPMasquerade=yes will make it forward too. Done. In a standard, understandable, uniform way. Unlike picking & choosing some random daemons to serve random jobs, unlike hacking up some random iptables script you've duck taped together.
I love love love that we're making videos to show people the basics of compute. Adore it. But wow this is souch less ad-hoc & souch more orderly & easier with systemd.
Well, it does indeed look like an elegant solution to the problem. Why don't you sum it up nicely in a blog post and submit it to HN?