Mainline Linux on the MikroTik RB3011(earth.li)
earth.li
Mainline Linux on the MikroTik RB3011
https://www.earth.li/~noodles/blog/2020/09/rb3011-mainline.html
40 comments
That seems fairly mainline to me, as opposite to say μClinux.
μClinux was merged into mainline more than a decade ago.
I think the reason it's called "mainline" is due to the fact that it's based on vanilla Linux kernel even if you need to recompile the mainline kernel with the modified kernel modules. It is not based on something of different kernel branch or fork, for examples OpenWRT or Linux Switching Appliance (LISA) that it's not from the mainline [1].
If by any chance that Jonathan is reading this, mainline Linux kernel that can be run on the cheap Banana Pi router (BPI-R64 price USD$64) will be very handy and useful [2].
[1]http://lisa.mindbit.ro/
[2]http://www.banana-pi.org/r64.html
If by any chance that Jonathan is reading this, mainline Linux kernel that can be run on the cheap Banana Pi router (BPI-R64 price USD$64) will be very handy and useful [2].
[1]http://lisa.mindbit.ro/
[2]http://www.banana-pi.org/r64.html
It's not as cheap, but you might have better luck with something based off the Espressobin platform, although I can't remember if the switch chip is completely closed source.
Armbian on Espressobin is a nice platform. I have three v5 Espressobins. Sadly, the v5 units have some quality issues. One of the three runs non-stop. The second reboots sporatically. The third freezes sporatically and has to be unplugged. I don't know if the cause is thermal or poor soldering or something else. I would buy many more if they were reliable.
That's a pity that the build quality is bad. Librecomputer might be worth a look - they spent years and significant amounts of money upstreaming the kernel for at least one of their boards, but I haven't used them enough to comment on the build quality.
I had some experience with Espressobin v5 and v7 devices. All the issues that I had with the devices were caused by terrible software support. Openwrt was unstable. The original kernel was too old.
The best results were using armbian kernel and userspace for this device. You should be able to build one in a less than an hour on modern pc.
I have a v7 device that I use with a sata drive. It has been stable so far.
Edit: forgot about needing a recent bootloader. This is also mentioned by armbian docs.
The best results were using armbian kernel and userspace for this device. You should be able to build one in a less than an hour on modern pc.
I have a v7 device that I use with a sata drive. It has been stable so far.
Edit: forgot about needing a recent bootloader. This is also mentioned by armbian docs.
"It is not based on something of different kernel branch or fork, for examples OpenWRT or Linux Switching Appliance (LISA)..."
Ah, yes, that makes sense. He started with Debian. But did do a lot of work with modules, drivers, device trees, the boot loader, etc.
Ah, yes, that makes sense. He started with Debian. But did do a lot of work with modules, drivers, device trees, the boot loader, etc.
Mainline linux is often used in the embedded Linux community to describe a kernel downloaded from the official repository rather than an old, patched-to-death one provided by the SoC vendor.
Having prepared embedded devices myself, it is quite common to patch the kernel you are working with to work around quirks in your hardware. However, it is much more pleasant to do so on a mainline kernel than on something that was already patched and never maintained.
Also note that the author mentions that he is currently in the process of getting his patches merged in Linux, and that most of the work should be included in 5.9. So perhaps not mainline yet, but future mainline definitely.
Having prepared embedded devices myself, it is quite common to patch the kernel you are working with to work around quirks in your hardware. However, it is much more pleasant to do so on a mainline kernel than on something that was already patched and never maintained.
Also note that the author mentions that he is currently in the process of getting his patches merged in Linux, and that most of the work should be included in 5.9. So perhaps not mainline yet, but future mainline definitely.
As I read this, I can't help but think of how much easier it would be to get a "regular" Linux distribution running on most of the MikroTik routers if they would upstream -- or release or even publish -- their changes, kernel configurations, and such.
Instead, they're basically selling devices running a "proprietary" Linux and, IMO, leeching off of the kernel development community and others.
Fortunately, the OpenWRT community has, over the years, managed to get OpenWRT running on many of the MikroTik devices. (Occasionally, depending on the particular device, there's a "performance hit" due to hardware acceleration that doesn't work without binary blobs from the vendor, for example, but they've certainly managed to make these devices for those of us who would prefer to not touch RouterOS!)
Instead, they're basically selling devices running a "proprietary" Linux and, IMO, leeching off of the kernel development community and others.
Fortunately, the OpenWRT community has, over the years, managed to get OpenWRT running on many of the MikroTik devices. (Occasionally, depending on the particular device, there's a "performance hit" due to hardware acceleration that doesn't work without binary blobs from the vendor, for example, but they've certainly managed to make these devices for those of us who would prefer to not touch RouterOS!)
What do you like about openWRT over routerOS?
For me, I get to run whatever I want instead of being limited to whatever mikrotik has implemented, eg CAKE QoS, Wireguard and dnscrypt-proxy. I also feel much more confident in the open source applications like dropbear and uhttpd than Mikrotiks proprietary implementations which have had numerous security flaws over the years.
I didn't know that open source software has no security flaws.
That’s not what GP said. Bad strawman :)
Short answer: choice, control, flexibility, and that warm, fuzzy feeling.
Very cool. I've been running openwrt on mikrotik rb2011* and rb750gl. Thing is the rb2011 has 5x1gbe and 5x10/100 + sfp. I think the rb3011 would be a worthy replacement.
Openwrt is really cool but it supports mostly wifi+4+1-port routers, and very few switches like the kind you'd run your lan with.
Openwrt is really cool but it supports mostly wifi+4+1-port routers, and very few switches like the kind you'd run your lan with.
> However the FTTP ONT meant I was using up an additional ethernet port on the router, and I was already short
You do not need an ONT if you have an SFP slot available. Get a suitable SFP module for your fiber type, and you can plug the fiber directly into your router.
This safes you a little bit of power, latency and space. Overall I find it to be a more elegant solution.
You do not need an ONT if you have an SFP slot available. Get a suitable SFP module for your fiber type, and you can plug the fiber directly into your router.
This safes you a little bit of power, latency and space. Overall I find it to be a more elegant solution.
Depends on the ONT. With GPON, it is not that simple. Many ISPs will refuse connecting anything but their ONT to their network.
And they have a point: the devices are not that compatible as they should be, and most of them will not work with their TRS069 system as the supplied CPE does.
And they have a point: the devices are not that compatible as they should be, and most of them will not work with their TRS069 system as the supplied CPE does.
I've seen commercial ISPs (Orang France for instance) provide SFP connectors to plug into their routers. This is nice, as you can just plug that SFP inside your own.
As an aside, I do not have a SFP-capable router, but I have a few SFP slots on my managed switch. I've been toying with the idea of putting one in a dedicated VLAN with the WAN port of my router, with another ethernet cable connecting to the LAN port. I might also get away with using only one ethernet port if using tagged VLANs, but that would limit the bandwidth.
As an aside, I do not have a SFP-capable router, but I have a few SFP slots on my managed switch. I've been toying with the idea of putting one in a dedicated VLAN with the WAN port of my router, with another ethernet cable connecting to the LAN port. I might also get away with using only one ethernet port if using tagged VLANs, but that would limit the bandwidth.
Orange France does that???
In my experience, Orange Slovakia refuses to do that; they have their Huawei OLTs, hand out Huawei ONTs and they only thing they were willing to do is switching the ONT to bridge mode.
That's not all: you cannot use any Huawei ONT, even if it is the same model as they use. You must use one provided by them. They are activated by serial numbers and they won't activate it if isn't their. So for a long time I had a router with SFP interface, but it was useless and I had to run the ONT next to it anyway.
On similar topic: how is IPTV done in France? Here, it looks like IGMP, the ONT does IGMP proxying and the upstream interface for the multicast subnet is a separate VLAN. (This is something, that many routers have problem with).
In my experience, Orange Slovakia refuses to do that; they have their Huawei OLTs, hand out Huawei ONTs and they only thing they were willing to do is switching the ONT to bridge mode.
That's not all: you cannot use any Huawei ONT, even if it is the same model as they use. You must use one provided by them. They are activated by serial numbers and they won't activate it if isn't their. So for a long time I had a router with SFP interface, but it was useless and I had to run the ONT next to it anyway.
On similar topic: how is IPTV done in France? Here, it looks like IGMP, the ONT does IGMP proxying and the upstream interface for the multicast subnet is a separate VLAN. (This is something, that many routers have problem with).
> Orange France does that???
I confirm they do. They have to support enterprise-lite setups on the same GPON as the standard consumer stuff.
I confirm they do. They have to support enterprise-lite setups on the same GPON as the standard consumer stuff.
I know, with Alcatel-Lucent-Nokia, you can get their SFP, and reflash their OMCI config vars, and LOID from their router into it.
If you can get config GPON OMCI strings from your router, maybe this will work for you.
If you can get config GPON OMCI strings from your router, maybe this will work for you.
> Orange France does that???
I've seen both, it might depend on the location, or other factors. Maybe you could ask explicitly?
See there: https://lafibre.info/orange-les-news/show-hello-2016/
That website also has a lot of information (in French) regarding ISP configs, and I think it would answer your question regarding IPTV. There is a whole section dedicated to replacing the box with a custom router: https://lafibre.info/remplacer-livebox/
It looks like there is a VLAN (840) for multicast TV, and another for VoD (838). There are a few vendor-specific field the DHCP server has to pass the decoder if you want that to work.
From one of the topics on that website, this English document is linked, which could be useful: https://docs.opnsense.org/manual/how-tos/orange_fr_tvf.html
I've seen both, it might depend on the location, or other factors. Maybe you could ask explicitly?
See there: https://lafibre.info/orange-les-news/show-hello-2016/
That website also has a lot of information (in French) regarding ISP configs, and I think it would answer your question regarding IPTV. There is a whole section dedicated to replacing the box with a custom router: https://lafibre.info/remplacer-livebox/
It looks like there is a VLAN (840) for multicast TV, and another for VoD (838). There are a few vendor-specific field the DHCP server has to pass the decoder if you want that to work.
From one of the topics on that website, this English document is linked, which could be useful: https://docs.opnsense.org/manual/how-tos/orange_fr_tvf.html
> I've seen commercial ISPs (Orang France for instance) provide SFP connectors to plug into their routers.
Same here. I use the SFP module that came with the router from my ISP.
I was unhappy with the quality of the ISP supplied router, and the fact that it ran a 6 year old custom image which was vulnerable to the KRACK attack. Since my ISP does not allow upgrading the firmware, I decided to ditch their router completely. I plugged my ISP's SFP module into my own Ubiquity device and it works flawless.
Same here. I use the SFP module that came with the router from my ISP.
I was unhappy with the quality of the ISP supplied router, and the fact that it ran a 6 year old custom image which was vulnerable to the KRACK attack. Since my ISP does not allow upgrading the firmware, I decided to ditch their router completely. I plugged my ISP's SFP module into my own Ubiquity device and it works flawless.
> I've been toying with the idea of putting one in a dedicated VLAN with the WAN port of my router, with another ethernet cable connecting to the LAN port.
That's what I'd like to do, but I haven't bothered to get a bi-directional SFP for my switch. Soon...
> I might also get away with using only one ethernet port ...
I would definitely avoid that at all costs! Exposing your internal network to your ISP's network is a really bad idea. With a managed switch, you'll want to disable as much of the "link-local" and other layer 2 "noise" as you can (e.g., CDP, LLDP, STP, etc.).
You may need to enable 802.1Q on your upstream-facing switchport anyways (and configure VLAN IDs, of course), but this would depend on your ISP's configuration. Either way, I'd still use a separate physical cable (between the router and switch) just for the "LAN side" -- if you weigh the pros and cons, it's a no-brainer!
That's what I'd like to do, but I haven't bothered to get a bi-directional SFP for my switch. Soon...
> I might also get away with using only one ethernet port ...
I would definitely avoid that at all costs! Exposing your internal network to your ISP's network is a really bad idea. With a managed switch, you'll want to disable as much of the "link-local" and other layer 2 "noise" as you can (e.g., CDP, LLDP, STP, etc.).
You may need to enable 802.1Q on your upstream-facing switchport anyways (and configure VLAN IDs, of course), but this would depend on your ISP's configuration. Either way, I'd still use a separate physical cable (between the router and switch) just for the "LAN side" -- if you weigh the pros and cons, it's a no-brainer!
> I would definitely avoid that at all costs! Exposing your internal network to your ISP's network is a really bad idea.
I have yet to see a single ISP that doesn't do it on their own side.
I have yet to see a single ISP that doesn't do it on their own side.
Hmm, I think it would not necessarily expose the rest of the network, though: if the router supports it, I would use a tagged vlan (let's say wan.42) as the upstream connection, and wan.10 as a lan port, part of the switched interfaces.
On the switch side, I'd have put the SFP connector inside VLAN 42, untagged, and put every other port in VLAN 10, untagged.
There are probably a couple other ways to achieve this, and ISPs sometimes use tagged VLANs as well. But I think that could work (I have little experience with VLANs, so I might be wrong, feel free to point it out to me).
On the other hand, if I had gigabit fiber, a single cable would halve my bandwidth. And I'd pick the multiple cable option over the other as long as I can afford it (enough ports)!
Thanks for pointing out the layer 2 "noise". I wonder if my switch wouldn't support bridging together two ports, I think it runs Linux...
On the switch side, I'd have put the SFP connector inside VLAN 42, untagged, and put every other port in VLAN 10, untagged.
There are probably a couple other ways to achieve this, and ISPs sometimes use tagged VLANs as well. But I think that could work (I have little experience with VLANs, so I might be wrong, feel free to point it out to me).
On the other hand, if I had gigabit fiber, a single cable would halve my bandwidth. And I'd pick the multiple cable option over the other as long as I can afford it (enough ports)!
Thanks for pointing out the layer 2 "noise". I wonder if my switch wouldn't support bridging together two ports, I think it runs Linux...
> Depends on the ONT. With GPON, it is not that simple. Many ISPs will refuse connecting anything but their ONT to their network.
Having this exact issue in a place where I got stuck now.
GPON hardware makers are artificially limiting interoperability, while the underlying hardware is fully capable of that.
GPON has a out of band configuration protocol which is usually unavailable to the OS. Here they put their vendor ID block.
Huawei is a virtual monopoly in GPON, and they strike such deals where they block competitors in exchange for discount left, and right.
GePON is by far better in this regard as it had no technical provisions for anything like this to start with.
Having this exact issue in a place where I got stuck now.
GPON hardware makers are artificially limiting interoperability, while the underlying hardware is fully capable of that.
GPON has a out of band configuration protocol which is usually unavailable to the OS. Here they put their vendor ID block.
Huawei is a virtual monopoly in GPON, and they strike such deals where they block competitors in exchange for discount left, and right.
GePON is by far better in this regard as it had no technical provisions for anything like this to start with.
Some Mikrotik devices (not the RB3011 unfortunately) support metarouter which allows you to run additional RouterOS or Linux instances as virtual machines[0].
[0] https://wiki.mikrotik.com/wiki/Manual:Metarouter
[0] https://wiki.mikrotik.com/wiki/Manual:Metarouter
I’ve been using RouterOS on a rb2011 since 2014, and it’s been a great device. Only wish I could have something like Pi Hole running on it, rather than a separate device.
Not sure if I would ever try to install something else and loose hardware acceleration.
Not sure if I would ever try to install something else and loose hardware acceleration.
This is great. Tempted to try this on a 4011 with the 10G SFP+ port. Mikrotik makes some awesome equipment for the price.
it's realy great for the price
i recently picked up Mikrotik hEX S https://www.amazon.co.uk/gp/product/B07F7HDRKX/ref=ppx_yo_dt...
it has PoE input and output, and routerOS has enough features for me.
it has PoE input and output, and routerOS has enough features for me.
I was wondering if something like this existed. Thank you.
so what is the current best linux based OS for a router/firewall.
pfsense grade really. I have heard about vyos, but nothing more. Given that bpfilter & XDP is now in the kernel, i was wondering if there's something built on top of the latest and greatest.
pfsense grade really. I have heard about vyos, but nothing more. Given that bpfilter & XDP is now in the kernel, i was wondering if there's something built on top of the latest and greatest.
> ... linux based OS ...
The differences WRT kernel between various Linux distributions are likely not significant enough to matter for the overwhelming majority of users.
Unless you have "non-typical requirements", pretty much any of them will almost certainly be "good enough".
(Personally, my preferences are OpenBSD, FreeBSD, and OPNsense, in that order, but I'm weird. If I had to choose something built on Linux, OpenWRT would probably be the one that I'd reach for "by default".)
The differences WRT kernel between various Linux distributions are likely not significant enough to matter for the overwhelming majority of users.
Unless you have "non-typical requirements", pretty much any of them will almost certainly be "good enough".
(Personally, my preferences are OpenBSD, FreeBSD, and OPNsense, in that order, but I'm weird. If I had to choose something built on Linux, OpenWRT would probably be the one that I'd reach for "by default".)
+1 for OPNsense if you're willing to deviate from Linux and get a great management web console. I've been running it at home for over a year without any hiccups.
Oh yes awesome, I tried this on a CRS326 but the main issue is the lack of storage
How does a router differ from a managed switch?
Unless you mean a layer 3 switch a managed switch usually provides port level configurations. Such as VLANs and QoS features to prioritize VOIP or other traffic. The router will still control the network and a decent SDN controller can inform the network without having to perform device level configs but I digress.
With a layer 3 switch you can avoid hops to the router and also get some hardware acceleration. Although the lines are increasingly blurred between switch and router with hardware acceleration becoming popular all around. Most consumer routers are router/switch combos.
With a layer 3 switch you can avoid hops to the router and also get some hardware acceleration. Although the lines are increasingly blurred between switch and router with hardware acceleration becoming popular all around. Most consumer routers are router/switch combos.
Kudos to him though, it looks like it would be a popular device if it becomes easier to get Linux running well on it.