Securing a Linux Server(kenhv.com)
kenhv.com
Securing a Linux Server
https://kenhv.com/blog/securing-a-linux-server
26 comments
Is there any point in Fail2ban if you're using keys and have disabled passwords? I guess defense in depth and all that
> Is there any point in Fail2Ban if you're using keys and have disabled passwords?
I’d say no. Back in the mid 2000s, I used to use Fail2Ban as an extra layer of defense but users on Stack Exchange and Hacker News (like tptacek¹) convinced me that it was pointless if I’d already disabled password authentication.
To minimise noise in my logs and to have an extra layer of defense, I only allow TCP access to Port 22 (with rate-limiting) from my home ISP’s network block, my work IP address and via the Wireguard network interface (in case my home or work ISP change the IP addresses they provide to customers).
I have considered using Fail2Ban to stop spammers using too much Postfix resources but so far I’ve got away with postscreen and configuring Postfix to reject spam attempts as early as possible during the SMTP transaction. Similarly, my Apache server gets hammered with exploit attempts but I haven’t got around to investigating how useful Fail2Ban would be for minimising how much server resources are used in responding to these malicious HTTP requests.
¹ https://news.ycombinator.com/item?id=37795100#37796639
I’d say no. Back in the mid 2000s, I used to use Fail2Ban as an extra layer of defense but users on Stack Exchange and Hacker News (like tptacek¹) convinced me that it was pointless if I’d already disabled password authentication.
To minimise noise in my logs and to have an extra layer of defense, I only allow TCP access to Port 22 (with rate-limiting) from my home ISP’s network block, my work IP address and via the Wireguard network interface (in case my home or work ISP change the IP addresses they provide to customers).
I have considered using Fail2Ban to stop spammers using too much Postfix resources but so far I’ve got away with postscreen and configuring Postfix to reject spam attempts as early as possible during the SMTP transaction. Similarly, my Apache server gets hammered with exploit attempts but I haven’t got around to investigating how useful Fail2Ban would be for minimising how much server resources are used in responding to these malicious HTTP requests.
¹ https://news.ycombinator.com/item?id=37795100#37796639
I move SSH to a different port... I know it's easy enough to discover through a port scan, but it cuts a LOT of noise down. For home, I only allow the wireguard port from the outside.
I tend to start with Ubuntu Server these days as the SSH config is pretty much where I want in the box and will import my public key during setup. I also now use Caddy for reverse-proxy duties over Nginx.
I tend to start with Ubuntu Server these days as the SSH config is pretty much where I want in the box and will import my public key during setup. I also now use Caddy for reverse-proxy duties over Nginx.
Instead of rate limiting, more than a decade ago I started simply changing SSH port from 22 to something else.
In case I needed to proxy through home, or access something like web ui for home heating system - I simply used ssh tunnel (socks localhost etc)...
Of course now all that is simply done via WireGuard/Tailscale/etc.
In case I needed to proxy through home, or access something like web ui for home heating system - I simply used ssh tunnel (socks localhost etc)...
Of course now all that is simply done via WireGuard/Tailscale/etc.
I'd even go one step further and say portknocking would do a lot of good too, in addition to changing the default SSH port.
If anyone actually does this - please have a backup for the time when either Fail2Ban or/and PortKnocking (so both) fail or lock you out.
I've never had knockd fail me, can you fill me in on what pitfall got yours to go down? Just in case.
Besides dummy stuff like wrong config of knockd itself - which hopefully gets ironed out during initial (local/on-prem setup).
Every now and then particular knock ports might be blocked (DROP) by cliënt side ISP/wifi/etc you happen to be using at the moment.
Or your fiber optic connection package gets a speed boost, along with being moved behind CGNAT.
OK in my case switching from cable to fiber already included ending up behind a CGNAT, but people already on that ISP actually got that 2in1 surprise.
Every now and then particular knock ports might be blocked (DROP) by cliënt side ISP/wifi/etc you happen to be using at the moment.
Or your fiber optic connection package gets a speed boost, along with being moved behind CGNAT.
OK in my case switching from cable to fiber already included ending up behind a CGNAT, but people already on that ISP actually got that 2in1 surprise.
I put ssh on ipv6 only on a cloud vps I was using. Crickets. No one ever noticed it was there.
Oh they knew. Just no one has IPv6 to connect.
Fail2Ban doesn't do much for SSH other than keeping your logs cleaner if you're using key based auth. It's quite good for protecting other services like Vaultwarden for example. Of course, it's just one additional layer. The important part is to configure the services themselves to be more secure.
Fail2ban can also block ips for other services that may be listening beyond just ssh.
I suppose that helps recoup some system load that otherwise would be wasted in lengthy random port/service poking
There are a few other posts on HN with the same title. Some things to also consider that I had not seen mentioned:
PCI
CIS
Etc…
Include many more things specifically around ssh that you can do outside of fail2ban, also things that are requirements for the above….
These posts are good but slightly miss a lot of security practices that are “standard”. As always the best security is not allowing the system to be connected to anything. But in the event that you have to have a system with such availability, it’s always best to introduce at least CIS foundations and whatever you see fit for security. Just my .02..
Include many more things specifically around ssh that you can do outside of fail2ban, also things that are requirements for the above….
These posts are good but slightly miss a lot of security practices that are “standard”. As always the best security is not allowing the system to be connected to anything. But in the event that you have to have a system with such availability, it’s always best to introduce at least CIS foundations and whatever you see fit for security. Just my .02..
I have received a lot of feedback regarding this. I'm waiting for Ubuntu to update their CIS docs for 24.04, I'll update my post when they do. I keep a lot of my blog posts regularly updated, this post will be one of them.
This is neat. Modern take and very pragmatic.
Modern take would be to simply not open anything to the outside world - except WireGuard (TailScale or such).
From there everything is either considered "localhost" or a local network.
You can setup one or two central boxes (actual home lab "server" where you already have HTTP based services, and a raspberry pi zero 2 for backup) with TailScale.
With remote devices (including phones) in same tailscale network - you can access anything in home network as if you're physically home (but also have ACLs for kids/friends/etc).
On the other (professional) end - well then NginX and SSH are not even on the same network interface. And you run NginX LB/ReverseProxy on separate boxes compared to where actual apps/websites are ...etc.
From there everything is either considered "localhost" or a local network.
You can setup one or two central boxes (actual home lab "server" where you already have HTTP based services, and a raspberry pi zero 2 for backup) with TailScale.
With remote devices (including phones) in same tailscale network - you can access anything in home network as if you're physically home (but also have ACLs for kids/friends/etc).
On the other (professional) end - well then NginX and SSH are not even on the same network interface. And you run NginX LB/ReverseProxy on separate boxes compared to where actual apps/websites are ...etc.
This setup is the most secure, but it's also the most limiting - it's feasible only if you're hosting services for yourself or a couple of people.
Wouldn't that violate the concept of zero trust?
Which "zero trust" are you thinking off?
In case of "zero trust network" the answer is no it doesn't violate.
With WireGuard or TailScale/CloudFlare/etc you still know/verify identity of every person/device that has access to the (virtual and through it real) network.
In case of "zero trust network" the answer is no it doesn't violate.
With WireGuard or TailScale/CloudFlare/etc you still know/verify identity of every person/device that has access to the (virtual and through it real) network.
I'm using similar approach but with ZeroTier.
After following this guide, all requests to my website time out, so I guess it's secure!
Seems like it's something as simple as a missing `sudo ufw allow 'Nginx Full'` ?
It might be a firewall issue or one of the Fail2Ban jails. If you're using all the Nginx jails, try disabling them and see if it fixes the issue.
I'm surprised my post made it to HN :D
[deleted]