Ask HN: What are some fun projects to try out on a spare Linux file server?
I installed Kubuntu on an old PC and am currently using it as a remote dev environment & FTP server. Any other cool use-cases for a spare Linux box?
142 comments
Check these out:
- reddit.com/r/homelab
- reddit.com/r/homeserver
I'd say the content on /r/homelab is about 33/33/33 distributed between
1. Masturbatory massive rigs at your home that consume thousands of watts of power but look cool and its fun to say you have a 10-node vSAN cluster next to your dehumidifier.
2. People who like to pirate content like its their job - setting up Plex and all the newsgroup/torrent apps to auto-pirate shows and movies. For a 13 year old, that can be cool. For an adult who clearly has the cash to spend on non-essential items and in a world where time is money: I do not get it.
3. People with an old PC running Kubuntu for a remote dev / FTP environment who want to do more cool shit with their gear!
So as long as you enter the homelab reddit knowing this, you will have a better experience there.
I definitely second Pihole. I might suggest experimenting with virtualization: so you can turn that single computer into 5 or 6. Tools like Proxmox or even just installing KVM and managing it with (https://virt-manager.org/). You could Also look into running your own installation of Gitlab (although due to the footprint of the install, I might suggest keeping that isolated to a virtual machine)
- reddit.com/r/homelab
- reddit.com/r/homeserver
I'd say the content on /r/homelab is about 33/33/33 distributed between
1. Masturbatory massive rigs at your home that consume thousands of watts of power but look cool and its fun to say you have a 10-node vSAN cluster next to your dehumidifier.
2. People who like to pirate content like its their job - setting up Plex and all the newsgroup/torrent apps to auto-pirate shows and movies. For a 13 year old, that can be cool. For an adult who clearly has the cash to spend on non-essential items and in a world where time is money: I do not get it.
3. People with an old PC running Kubuntu for a remote dev / FTP environment who want to do more cool shit with their gear!
So as long as you enter the homelab reddit knowing this, you will have a better experience there.
I definitely second Pihole. I might suggest experimenting with virtualization: so you can turn that single computer into 5 or 6. Tools like Proxmox or even just installing KVM and managing it with (https://virt-manager.org/). You could Also look into running your own installation of Gitlab (although due to the footprint of the install, I might suggest keeping that isolated to a virtual machine)
> 2. People who like to pirate content like its their job - setting up Plex and all the newsgroup/torrent apps to auto-pirate shows and movies. For a 13 year old, that can be cool. For an adult who clearly has the cash to spend on non-essential items and in a world where time is money: I do not get it.
It took me about an afternoon or two to set up, was fun to do, and now my favorite TV shows just appear in Plex for me to watch whenever I want to. Half of the time they're not available on Netflix either, so I'd have to figure out what streaming provider they're on, deal with their special websites, not get notifications when a new episode airs, etc. It's just a better experience.
It took me about an afternoon or two to set up, was fun to do, and now my favorite TV shows just appear in Plex for me to watch whenever I want to. Half of the time they're not available on Netflix either, so I'd have to figure out what streaming provider they're on, deal with their special websites, not get notifications when a new episode airs, etc. It's just a better experience.
Or the case of discovering a great show on Netflix/HBO, binging all available seasons, only to discover that 4 more seasons have aired, and Netflix/HBO just didn’t bother enough to renew the seasons.
Or show is only available on the Nth $9.99/mo streaming platform, and I already pay for 3 others. Normally I sign up for a trial with a fake email address (getnada is great!), but some require a (valid) credit card, and I don’t have an unlimited supply of those.
Or the show simply isn’t available in your geographic region.
Or show is only available on the Nth $9.99/mo streaming platform, and I already pay for 3 others. Normally I sign up for a trial with a fake email address (getnada is great!), but some require a (valid) credit card, and I don’t have an unlimited supply of those.
Or the show simply isn’t available in your geographic region.
Revolut has "virtual cards" which are invalid after the first charge.
Only for paying customers. It used to be three free disposable cards per customer on the free tier but I woke up to find only virtual card left and no disposable ones. It's a shame these kinds of features are limited to these somewhat-sketchy online banks. None of the banks in my motherland offer digital disposable card numbers yet.
Some people just need to find additional validation to prove to themselves they're doing the right thing. Don't worry about it.
I have no idea what they're talking about though. Installing qBittorrent takes about a minute, adding an RSS feed one minute more; that's it. I think I configured my setup about two years ago, the files just magically appear in whatever directory I asked them to.
How much more time does this consume compared to opening the web browser, going to <whatever streaming site>, and so on? I'd say, about 300% less.
Edit: to whoever is reading this, after pressing the downvote/flag button, please do the following: set up a VPN to any third-world country, go to your favorite streaming provider, and have a gander at the breadth and diversity of their catalog.
I have no idea what they're talking about though. Installing qBittorrent takes about a minute, adding an RSS feed one minute more; that's it. I think I configured my setup about two years ago, the files just magically appear in whatever directory I asked them to.
How much more time does this consume compared to opening the web browser, going to <whatever streaming site>, and so on? I'd say, about 300% less.
Edit: to whoever is reading this, after pressing the downvote/flag button, please do the following: set up a VPN to any third-world country, go to your favorite streaming provider, and have a gander at the breadth and diversity of their catalog.
Yeah but isn't most of torrenting these shows illegal?
Also worth checking out is reddit.com/r/selfhosted, which tends to be much less 'masturbatory' than /r/homelab
> reddit.com/r/selfhosted
I second this, very laid back with good information.
I second this, very laid back with good information.
I didn't know about that sub. Thanks.
I massively don't get /r/homelab. I can't imagine having literal work as a hobby. Low-power stuff like Synologies and NUCs are cool, but all I see on the sub is huge power-wasters, and people even brag about it!
[1] https://old.reddit.com/r/homelab/comments/bshlr9/the_real_co...
[1] https://old.reddit.com/r/homelab/comments/bshlr9/the_real_co...
> "I can't imagine having literal work as a hobby. "
Think of it the other way. Instead of taking work home with them to create a hobby, maybe they found some way to get paid for their hobby. Maybe this is fun for them and getting paid for it during the day is just icing on the cake.
I don't really know though. If that's true, I'm jealous of them.
Think of it the other way. Instead of taking work home with them to create a hobby, maybe they found some way to get paid for their hobby. Maybe this is fun for them and getting paid for it during the day is just icing on the cake.
I don't really know though. If that's true, I'm jealous of them.
When I found that sub, I had hoped it would be a bit more hacky. Like this is how I build a vSAN cluster out of a bunch of Mac Minis the office was throwing out.
Instead it is just people who buy an bunch of old enterprise gear on eBay. Though every once in a while someone posts something really cool, which is the only reason I stay subbed. Like a little while back someone figured out how to add 10Gb Ethernet to a NUC.
I have a couple of old Dell T320s that I bought for a project last year. I finished the project and powered them off because they make my office too hot. I really should think about selling them.
I am getting ready to to build a cluster out of Raspberry PI 4s as there is some cloud services that I am paying for that I want to host at home. At least those are pretty power efficient.
Instead it is just people who buy an bunch of old enterprise gear on eBay. Though every once in a while someone posts something really cool, which is the only reason I stay subbed. Like a little while back someone figured out how to add 10Gb Ethernet to a NUC.
I have a couple of old Dell T320s that I bought for a project last year. I finished the project and powered them off because they make my office too hot. I really should think about selling them.
I am getting ready to to build a cluster out of Raspberry PI 4s as there is some cloud services that I am paying for that I want to host at home. At least those are pretty power efficient.
I'm on homelab. It's for fun.
It's the same reason why we have cars that have hundreds of horsepower when you can't drive above 65 mph on most highways and even accelerating fast can get you a ticket for "exhibition of speed". It's for fun, and a lot of the people on homelab use it to learn for their job.
It's the same reason why we have cars that have hundreds of horsepower when you can't drive above 65 mph on most highways and even accelerating fast can get you a ticket for "exhibition of speed". It's for fun, and a lot of the people on homelab use it to learn for their job.
2. = Buys CD, gets no FLAC: choices are as follows; 1.) dl FLAC rip 2.) waste my time repeating work someone has already done
I'm always going to pick option 1; guess I'll never be an adult :|
I'm always going to pick option 1; guess I'll never be an adult :|
Nah what you're doing is closer to r/datahoarder . It's legally grey, but a lot of people wouldn't really consider is real "piracy" in your case if you purchased a CD. I more consider it datahoarding in your case.
What he's talking about is all those people with 160TB+ servers that do nothing but seed torrents of movies and tv shows they may or may not even watch.
What he's talking about is all those people with 160TB+ servers that do nothing but seed torrents of movies and tv shows they may or may not even watch.
Format shifting is currently illegal in the UK. So even ripping a CD you own to flag is against the law.
(A civil thing, not a criminal offence).
(A civil thing, not a criminal offence).
So backups are against the law in the UK? How does that work?
Note: You can digitally "rip" an audio CD as a digital file; technically you are making a backup of the data that forms the audio on the CD. Provided you are only doing this for your personal use (not sharing), and you keep the original (that's an important part) - I can't see how such a law would even be enforceable?
Furthermore - what if you have old software from say, an IBM/370 system - that your business needs? You don't have that system any longer because the pieces all died, and they went to a museum, but you still rely on the software written in COBOL, so you run it on an emulator on a PC...
...obviously, the software is no longer running off of tapes or punchcards or whatever - ie "format shifting" - does this law cover that scenario too? "Sorry, chap, you'll just have to stuff your business because your antique computer died, but them's the breaks, mate?" - my sincere apologies... :)
Seriously, though - it just seems a trifle limiting if that's the case - ultimately preventing the use of backups and shifting of those backups onto future media (can you copy a floppy to your hard drive and run it from there?)...
Here in the United States, backups are allowed, of any media; but I do think there is a technical limitation in that you have to own the original copy, and you have to make the backup yourself. Thus, a copy of a ROM for an emulator would be a-ok, as long as you own the original cartridge, plus the original system it runs on - but you have to pull that ROM image from the cartridge (and any images needed for the emulated system from the original system) yourself - a copy from another source is not "legal".
In practice, nobody does this, though - but it probably could be checked for via serial numbers in the ROMs, if they wanted to enforce it.
Note: You can digitally "rip" an audio CD as a digital file; technically you are making a backup of the data that forms the audio on the CD. Provided you are only doing this for your personal use (not sharing), and you keep the original (that's an important part) - I can't see how such a law would even be enforceable?
Furthermore - what if you have old software from say, an IBM/370 system - that your business needs? You don't have that system any longer because the pieces all died, and they went to a museum, but you still rely on the software written in COBOL, so you run it on an emulator on a PC...
...obviously, the software is no longer running off of tapes or punchcards or whatever - ie "format shifting" - does this law cover that scenario too? "Sorry, chap, you'll just have to stuff your business because your antique computer died, but them's the breaks, mate?" - my sincere apologies... :)
Seriously, though - it just seems a trifle limiting if that's the case - ultimately preventing the use of backups and shifting of those backups onto future media (can you copy a floppy to your hard drive and run it from there?)...
Here in the United States, backups are allowed, of any media; but I do think there is a technical limitation in that you have to own the original copy, and you have to make the backup yourself. Thus, a copy of a ROM for an emulator would be a-ok, as long as you own the original cartridge, plus the original system it runs on - but you have to pull that ROM image from the cartridge (and any images needed for the emulated system from the original system) yourself - a copy from another source is not "legal".
In practice, nobody does this, though - but it probably could be checked for via serial numbers in the ROMs, if they wanted to enforce it.
IIRC/IANAL/etc, in the US backups in and of themselves are not infringement, but transferring tools that break copyright protection is illegal. So, you can back up your DVD, but you can't transfer (upload, download, etc) tools that break the encryption. So, unless you broke the encryption yourself independently, you're still not in the clear.
The IBM/370 system example seems pretty farfetched. Especially since they said format shifting was a civil thing, not criminal.
In general, there aren't a lot of clear cut laws that state "you can do X, but not Y". Mostly because that's just not how our legal system works. Most of the time, it's more about your intent rather than what you technically did.
The most clear example of this is the differences between first degree murder, second degree murder, and manslaughter. All different crimes with different punishments, but ultimately you technically killed someone.
Another more relevant example is how cybercrimes are treated. Just because a system let you have access to a file does not mean that it was legal access. Otherwise, none of them would be enforceable since it's always a machine granting you access (that you shouldn't have). It's all about your intent.
Usually, if you really are just making your own backups from your own copies, you don't have much to worry about since, at least for now, it's not really enforceable.
The IBM/370 system example seems pretty farfetched. Especially since they said format shifting was a civil thing, not criminal.
In general, there aren't a lot of clear cut laws that state "you can do X, but not Y". Mostly because that's just not how our legal system works. Most of the time, it's more about your intent rather than what you technically did.
The most clear example of this is the differences between first degree murder, second degree murder, and manslaughter. All different crimes with different punishments, but ultimately you technically killed someone.
Another more relevant example is how cybercrimes are treated. Just because a system let you have access to a file does not mean that it was legal access. Otherwise, none of them would be enforceable since it's always a machine granting you access (that you shouldn't have). It's all about your intent.
Usually, if you really are just making your own backups from your own copies, you don't have much to worry about since, at least for now, it's not really enforceable.
You can make backups. Ripping to mp3 to back something up is probably allowed. But most people ripping to mp3 are not backing up their CDs, they're format-shifting, and format-shifting is currently not allowed.
You're definitely not the kind of person I was singling out in my remarks. If you own something, it's a different story. In your case, owning a CD or ripping it is the same in my eyes.
OP is talking about torrenting someone else's rip, not ripping from the CD that they bought.
That means they downloaded torrenting software, accessed a tracker and knowingly joined a swarm of peers sharing illegally distributed content.
If your convoy "crashes the gate doin' 98," you don't get dispensation just because you happened to be driving a Chartreuse microbus[1].
[1] https://en.m.wikipedia.org/wiki/Convoy_(song)
That means they downloaded torrenting software, accessed a tracker and knowingly joined a swarm of peers sharing illegally distributed content.
If your convoy "crashes the gate doin' 98," you don't get dispensation just because you happened to be driving a Chartreuse microbus[1].
[1] https://en.m.wikipedia.org/wiki/Convoy_(song)
I learned a lot of *NIX stuff just by way of introduction through setting up a fileserver to host Plex, network shares, music server, etc. Doesn't really fall into any of your categories. It's more like:
4. People who upgraded their PC and have an old PC that is currently doing nothing (but could).
Sure I could have bought a NAS, but that would have cost more money and I wouldn't have learned nearly as much. And I wouldn't have been able to transcode streams.
4. People who upgraded their PC and have an old PC that is currently doing nothing (but could).
Sure I could have bought a NAS, but that would have cost more money and I wouldn't have learned nearly as much. And I wouldn't have been able to transcode streams.
pbk1, your comment appears to be dead. You may have tripped some anti-spam filter because your account is relatively new and you have posted many links in your comments.
Along with all other suggestions:
- Syncthing node https://syncthing.org ;
- WireGuard server https://wireguard.com ;
- Pi-hole or just lying DNS Server https://pi-hole.net, https://gitlab.com/moviuro/moviuro.bin/blob/master/lie-to-me ;
- Nextcloud https://nextcloud.com/ ;
- Your own little blog/site with https (LetsEncrypt), nginx, etc.
- Syncthing node https://syncthing.org ;
- WireGuard server https://wireguard.com ;
- Pi-hole or just lying DNS Server https://pi-hole.net, https://gitlab.com/moviuro/moviuro.bin/blob/master/lie-to-me ;
- Nextcloud https://nextcloud.com/ ;
- Your own little blog/site with https (LetsEncrypt), nginx, etc.
I'm really loving NextCloud. It's now my personal everything server interface, and runs on an unraid box in Docker:
- My calendar, to-do list (via CalDAV, and using GoodTask), and contacts are all hosted on the instance, including shared family calendars
- It's my "personal Dropbox" but with 12 TB of storage, and using OnlyOffice it's also my personal Google Docs with a web editor
- NextCloud's app syncs a complete copy of all my photos from my phone automatically (in addition to using iCloud Photo Library)
- I have a script that auto-imports all footage from my GoPro when I plug it in and makes it accessible via NextCloud, so I can share raw footage with other people. When I edit video, it's all via my iOS devices and LumaFusion connecting to the SMB server (which, granted, isn't NextCloud but I can preview all the files via the NextCloud interface)
- It's all backed up to the cloud as well
- It's accessible externally (I use Traefik for load balancing and LetsEncrypt for SSL)
- My calendar, to-do list (via CalDAV, and using GoodTask), and contacts are all hosted on the instance, including shared family calendars
- It's my "personal Dropbox" but with 12 TB of storage, and using OnlyOffice it's also my personal Google Docs with a web editor
- NextCloud's app syncs a complete copy of all my photos from my phone automatically (in addition to using iCloud Photo Library)
- I have a script that auto-imports all footage from my GoPro when I plug it in and makes it accessible via NextCloud, so I can share raw footage with other people. When I edit video, it's all via my iOS devices and LumaFusion connecting to the SMB server (which, granted, isn't NextCloud but I can preview all the files via the NextCloud interface)
- It's all backed up to the cloud as well
- It's accessible externally (I use Traefik for load balancing and LetsEncrypt for SSL)
Wireguard+NFS is the sort of setup I really wish I had years ago. It's so damn convenient to set up and to use.
NFS is kind of horrible. Last time I used it, I had hard lockups when I unplugged my machine from the network.
SSHFS however works wonders, and doesn't require the VPN at all.
SSHFS however works wonders, and doesn't require the VPN at all.
Last item (blog/site) could run afoul of ISP terms. I would double check before hosting anything public.
I have Spectrum and I just checked, it is Prohibited to:
> Running any type of server on the system that is not consistent with personal, residential use. This includes but is not limited to FTP, IRC, SMTP, POP, HTTP, SOCS, SQUID, NTP, DNS or any multi-user forums.
That said, I run a small stack of personal stuff, including a public resume-style site, and I've never heard a peep from them. Because there's nothing commercial I feel like I could make an argument my site is "residential".
I did have my server pwned once though (I came home and it was very clear from the fan intensity that it was mining bitcoin) so I'm not sure I'd recommend it to anyone.
> Running any type of server on the system that is not consistent with personal, residential use. This includes but is not limited to FTP, IRC, SMTP, POP, HTTP, SOCS, SQUID, NTP, DNS or any multi-user forums.
That said, I run a small stack of personal stuff, including a public resume-style site, and I've never heard a peep from them. Because there's nothing commercial I feel like I could make an argument my site is "residential".
I did have my server pwned once though (I came home and it was very clear from the fan intensity that it was mining bitcoin) so I'm not sure I'd recommend it to anyone.
My Synology luckily was never pwned, but there were millions of attempts. Noticed higher than usual traffic one day, opened the access logs and found someone attempting to brute-force the SSH credentials... unfortunately for them I don't use passwords, only certs for authentication.
I'm actually doing this right now, and it should be fine as long as you use a service that offers dynamic DNS. If your ISP changes your IP, the dynamic DNS service will automatically point to the new IP.
Namecheap offers this when you buy a domain through them.
Namecheap offers this when you buy a domain through them.
Maybe consider setting up postfix to self-host email for a domain.
That's in the TODO list. Though I'm still looking for a 100% technology-agnostic guide of what mail is, how it works, what I need, etc.
I have a freenas NAS at home. I mostly use it so that I can have one home directory (via nfs) on multiple computers. (Firefox sucks when doing this - if anyone works for mozilla - help). I want to extend this to open LDAP so that I can use one login to all computers (not so much for me, but the family).
I have cups installed so that my printer needs only be setup once to work for all my machines. I'm thinking about putting my scanner on it as well (ideally I'd figure out how to use the buttons on the scanner making a copier), but since I never use my scanner this isn't really likely.
As a file server this is the obvious place to put all your shared media, then use kodi https://kodi.tv/ or volumio https://volumio.org/ or some other media center to share your media to all rooms. Setting this up is next on my list of projects, but so far I haven't got it working. Actually I had it working at my last house, but it was too hard to use so we didn't - let this serve as a lesion: make sure not only that it works, but that it is convenient to use.
Using it as a central hub for your home automation is an idea. I've resisted so far because https://www.home-assistant.io/blog/2016/01/19/perfect-home-a... makes a good point: I can't come up with a use for home automation that isn't worse than what I have without it.
My fundamental idea is I have one powerful, reliable, always-on machine. Anything that I want to run all the time should be on it. My other computers are only turned on when I want to do something on them.
I have cups installed so that my printer needs only be setup once to work for all my machines. I'm thinking about putting my scanner on it as well (ideally I'd figure out how to use the buttons on the scanner making a copier), but since I never use my scanner this isn't really likely.
As a file server this is the obvious place to put all your shared media, then use kodi https://kodi.tv/ or volumio https://volumio.org/ or some other media center to share your media to all rooms. Setting this up is next on my list of projects, but so far I haven't got it working. Actually I had it working at my last house, but it was too hard to use so we didn't - let this serve as a lesion: make sure not only that it works, but that it is convenient to use.
Using it as a central hub for your home automation is an idea. I've resisted so far because https://www.home-assistant.io/blog/2016/01/19/perfect-home-a... makes a good point: I can't come up with a use for home automation that isn't worse than what I have without it.
My fundamental idea is I have one powerful, reliable, always-on machine. Anything that I want to run all the time should be on it. My other computers are only turned on when I want to do something on them.
To solve your problem with Firefox, you might want to find their directory and symlink it to somewhere on a local drive.
That makes setting up new systems a pain, and is user-hostile
Users need to create a new profile for each new computer I buy. Firefox sync sortof works but I shouldn't need to store this data externally if I don't want to - and if I do want to it should still auto setup.
Users need to create a new profile for each new computer I buy. Firefox sync sortof works but I shouldn't need to store this data externally if I don't want to - and if I do want to it should still auto setup.
Look for a cache directory in ~/.firefox - symlink it to tmp or mount a ramdisk over it.
I used to do this at my previous work. It’s frustrating that browser vendors keep caches like that.
I used to do this at my previous work. It’s frustrating that browser vendors keep caches like that.
I have a dozen users (my family and some friends who visit often enough...). While nothing compared to any corporate system, it is still enough that I don't know how to do this for everybody...
What does freenas buy you over something like "just debian with nfs"? Is ZFS the appeal?
ZFS is the main appeal.
FreeNAS has a nice package around getting it all setup. Now that I have a family I don't have time to fight with computers all the time.
I'm a BSD user at heart, from way back. I use both, but BSD (as opposed to GNU) feels right to my fingers.
All of the above are factors.
FreeNAS has a nice package around getting it all setup. Now that I have a family I don't have time to fight with computers all the time.
I'm a BSD user at heart, from way back. I use both, but BSD (as opposed to GNU) feels right to my fingers.
All of the above are factors.
Low power 'buntu home servers are my jam. I've got a few open-source projects that you can plug right into a LAMP stack that you might be interested in...
[1] https://github.com/zelon88/HRCloud2 [2] https://github.com/zelon88/HRConvert2 [3] https://github.com/zelon88/HRScan2
[1] https://github.com/zelon88/HRCloud2 [2] https://github.com/zelon88/HRConvert2 [3] https://github.com/zelon88/HRScan2
Running home servers, I'm always worried about power use:
300 Watt(average)/1000 * 24 hr = 7.2 kw/hr per day
7.2 * 30 * $0.14 (roughly tax, generation + distribution in my area) ≈ $30/month to run the machine.
300 Watt(average)/1000 * 24 hr = 7.2 kw/hr per day
7.2 * 30 * $0.14 (roughly tax, generation + distribution in my area) ≈ $30/month to run the machine.
Definitely something to consider, but unless you're running some old big iron hardware, 300 watts would be a very high average for a mostly idle server.
I have a nearly 10 year old HP Microserver with 4 SATA disks and the average usage is less than 50 watts.
I have a nearly 10 year old HP Microserver with 4 SATA disks and the average usage is less than 50 watts.
One reason why I love my NUC. I have it running everything under the sun and it plus my networking gear draws 18 W. To be fair I haven't checked when it's under heavy load, but the internet says it shouldn't be more than around 35 W.
You don't have to buy actual rackmount server hardware to run home servers, just PCs with a high core count will do and there are some nice low-power options out there like Xeon-D based motherboards and Coffee Lake (i7-8700T) mini-PCs that offer high thread count for less than 35W TDP.
AMD's upcoming Ryzen 9 3950X is also rather exciting if it performs as well as promised. 32 threads for peak 105W TDP is really more than even most home lab builders are likely to ever need.
AMD's upcoming Ryzen 9 3950X is also rather exciting if it performs as well as promised. 32 threads for peak 105W TDP is really more than even most home lab builders are likely to ever need.
$30 a month should not be a hardship for most; I think I spend more for fast food every week (I certainly spend way more than that for random ebay and amazon purchases each month - and let's not get into what I spend on my Jeep).
I can understand it being a concern if you are trying to run multiple machines like this (beowulf cluster?) - or if you just don't want to be wasting the power by leaving the machines running an idle...
...but really, at idle without anything happening, I doubt a system even pulls half that. The best way to find out would be to plug the machine into some kind of "kill-o-watt" monitor and check what it uses at idle vs under load.
I would guess that a relatively modern machine, with a basic-level CPU, running headless and using SSDs or laptop hard drives - or just one or two standard desktop drives - likely wouldn't pull more than 150 watts (or could be made to do so).
I can understand it being a concern if you are trying to run multiple machines like this (beowulf cluster?) - or if you just don't want to be wasting the power by leaving the machines running an idle...
...but really, at idle without anything happening, I doubt a system even pulls half that. The best way to find out would be to plug the machine into some kind of "kill-o-watt" monitor and check what it uses at idle vs under load.
I would guess that a relatively modern machine, with a basic-level CPU, running headless and using SSDs or laptop hard drives - or just one or two standard desktop drives - likely wouldn't pull more than 150 watts (or could be made to do so).
If you can get VPS for $5 a month, I kind of don't feel like running some old hardware at home is worth $30. Ofc depends on what you use it for. For NAS I would rather buy actual NAS which would have Plex and other stuff and would take a lot less in power than old computer and would be a lot better in terms of storing data.
I've got a 5 year old server with 2x8 core sandy bridge xeons, 32GB ram (I had 128GB total but went down to 32GB since it wasn't being used), an SSD for the OS, and 8 hard drives from 2-8TB each (~30TB total).
I idle at 100W +/- 10W, which is ~$10/month. Sure, I could get a VPS for $5/month (with much lower specs), but the massive amount of local storage plus the ability to easily host multiple VMs really outweighs the extra $5/month cost to have everything local.
I run a handful of VMs and docker containers, such as pfSense, emby (similar to plex), HAProxy, samba shares, snapraid+mergerfs for storage, etc. Worth it to me, might not be for others unless they're interested in server administration.
When this dies I'll probably downgrade to something smaller and shoot for <50 watts, but it's not worth it until it actually fails.
I idle at 100W +/- 10W, which is ~$10/month. Sure, I could get a VPS for $5/month (with much lower specs), but the massive amount of local storage plus the ability to easily host multiple VMs really outweighs the extra $5/month cost to have everything local.
I run a handful of VMs and docker containers, such as pfSense, emby (similar to plex), HAProxy, samba shares, snapraid+mergerfs for storage, etc. Worth it to me, might not be for others unless they're interested in server administration.
When this dies I'll probably downgrade to something smaller and shoot for <50 watts, but it's not worth it until it actually fails.
At least for me and I'm sure for some others it's a hobby and just kinda excepted the higher electricity cost as part of my hobby's expenses. And as the others said, at idle you'll be using a lot less power.
That said you can run home servers with WAYYYY less power usage if you prefer. Single board computers are good bets, like the Raspberry pi 4 only use a max of ~15watts. Repurposed laptops have low power usage too.
That said you can run home servers with WAYYYY less power usage if you prefer. Single board computers are good bets, like the Raspberry pi 4 only use a max of ~15watts. Repurposed laptops have low power usage too.
As others said, 300 watts is a ton of power, if you're actually using that much compute then you're probably getting your money's worth.
My little pizza box R210 II PFSense router uses 30 watts (even though the PSU is 250), once I turned all the BIOS power settings down.
My little pizza box R210 II PFSense router uses 30 watts (even though the PSU is 250), once I turned all the BIOS power settings down.
Right now I have a R720 fully loaded with a mix of 16 SSD/HDDs, multiple Fusion-IO cards, 128GB+ ECC RAM, 10Gig Ethernet and when not under load it draws < 200 watts.
If you get something like a R620 from ebay, assuming the US, you can get one that uses less power.
If you get something like a R620 from ebay, assuming the US, you can get one that uses less power.
My ancient 5-disk box uses an average of 60W.
Modern hardware should pull even less.
Modern hardware should pull even less.
Just to see what happens, it might be cool to find a way to safely (as in without compromising your local network) provide root read, write and execute access to the whole anonymous, public internet while still being able to fully inspect everything that people (and bots) do. Maybe you can set an "intranet" of virtual machines to also see how people/bots move around the network. Kind of like:
https://www.xkcd.com/350/
https://www.xkcd.com/350/
expect a bricked firmware and/or child porn repository within 72 hours
Ha. Would people really do that? I honestly don't know what to expect. Maybe bitcoin mining?
Anyway, if we're talking about fun, I can't think of anything more exciting than seeing what happens in this scenario.
I imagine the way you expose it makes a difference, too, in what kind of people you attract.
Anyway, if we're talking about fun, I can't think of anything more exciting than seeing what happens in this scenario.
I imagine the way you expose it makes a difference, too, in what kind of people you attract.
I don’t think you need to expose it—a root account with “password” as the password and open SSH on port 22 would do the trick.
With adequate traffic filtering/limitations and VM snapshots, could be interesting.
Buy a second internet connection from your ISP if you're going to do that.
For a while I ran a box as a combo LAMP and ZoneMinder server; I ended up shutting it down as I wasn't using the LAMP server much any longer, and ZoneMinder was overkill for my purposes...
...but if you've never played around with a security camera system like ZoneMinder allows for, it can be an interesting experience. Like I said, complete overkill for my home security camera system - but it would be perfect if you set it up like an appliance, on a beefy system - because it needs a lot of CPU to process frames and such - and you have a lot of cameras to work with.
You can hook up virtually any kind of camera; from a simple USB webcam (heck, you could probably even get a parallel port black and white quickcam to work if you were crazy enough), to a set of cameras on a multi-input video digitizer card, to a full-on professional level PTZ (pan-tilt-zoom) camera (with PTZ control!).
It is fairly complex to set up properly; one could easily create a business around it selling turnkey camera security appliances if one wanted to, I believe - though I don't think I've seen any such business like that (unlike how there are/were tons of companies doing VOIP using custom Asterisk systems).
The whole application is written (IIRC) in Python; I don't recall what it uses for the video processing (whether OpenCV or some other library). It uses multiple processes to handle everything (it isn't a monolithic system), so it is fairly robust. But as I mentioned, the more cameras/video streams you have (and the higher resolution those cameras are), the more CPU you need to process everything. I'm not sure if there is a hard limit, or a way to segregate things across multiple systems (ie - some kind of clustering?); I think you can specify a separate DB server...
Anyhow - it's something different; I had it at one time running on a PII-350 with 64 Mb of RAM years ago - the last machine, IIRC, was a P4-933 with 256 Mb of RAM - and I could process a couple of streams of 640x480 color video from a couple of IP cameras (all I needed for my home system). But that was the limit for that machine. A more modern machine would probably work much better.
...but if you've never played around with a security camera system like ZoneMinder allows for, it can be an interesting experience. Like I said, complete overkill for my home security camera system - but it would be perfect if you set it up like an appliance, on a beefy system - because it needs a lot of CPU to process frames and such - and you have a lot of cameras to work with.
You can hook up virtually any kind of camera; from a simple USB webcam (heck, you could probably even get a parallel port black and white quickcam to work if you were crazy enough), to a set of cameras on a multi-input video digitizer card, to a full-on professional level PTZ (pan-tilt-zoom) camera (with PTZ control!).
It is fairly complex to set up properly; one could easily create a business around it selling turnkey camera security appliances if one wanted to, I believe - though I don't think I've seen any such business like that (unlike how there are/were tons of companies doing VOIP using custom Asterisk systems).
The whole application is written (IIRC) in Python; I don't recall what it uses for the video processing (whether OpenCV or some other library). It uses multiple processes to handle everything (it isn't a monolithic system), so it is fairly robust. But as I mentioned, the more cameras/video streams you have (and the higher resolution those cameras are), the more CPU you need to process everything. I'm not sure if there is a hard limit, or a way to segregate things across multiple systems (ie - some kind of clustering?); I think you can specify a separate DB server...
Anyhow - it's something different; I had it at one time running on a PII-350 with 64 Mb of RAM years ago - the last machine, IIRC, was a P4-933 with 256 Mb of RAM - and I could process a couple of streams of 640x480 color video from a couple of IP cameras (all I needed for my home system). But that was the limit for that machine. A more modern machine would probably work much better.
I've got an old HP laptop running a pihole-alike, boinc, syncthing, and a webcam capture from my window: https://johnrockefeller.net/guelph-webcam.
In addition, I also host local files in a temporary way so other machines in my home network can pick them up easily with `php -s`.
Running a home server on an old laptop is very useful. If you're worried about power usage, just set the CPU to the lowest MHz like I did.
If you're doing it, please don't forget to look at boinc, because they need all the help they can get with computing power.
In addition, I also host local files in a temporary way so other machines in my home network can pick them up easily with `php -s`.
Running a home server on an old laptop is very useful. If you're worried about power usage, just set the CPU to the lowest MHz like I did.
If you're doing it, please don't forget to look at boinc, because they need all the help they can get with computing power.
I've had a ton of fun with setting up FreeIPA [1] to do LDAP/Kerberos for single sign-on. It's a primarily RH project but I think they have a working server implementation for Ubuntu and derivatives.
[1]https://www.freeipa.org/page/Main_Page
[1]https://www.freeipa.org/page/Main_Page
I set this up 4 or 5 years ago, has it gotten easier since then? Specifically TOTP was rough.
Try out the free version of the Univention server. It’s pretty easy with ldap and active directory running out of the box, but you have to KVM it.
1: https://www.univention.com/products/ucs/
1: https://www.univention.com/products/ucs/
DNS tunnel "server" using iodine https://github.com/yarrick/iodine
"Free" Wifi in the whole world after that as most AP allow DNS requests.
"Free" Wifi in the whole world after that as most AP allow DNS requests.
I tried to do this about a year ago on an Ubuntu DO Droplet with a static public IP to point the NS record to and I never could get it to work. Even tried this fork which has a few more commits than upstream: https://github.com/advoryanskiy/iodine
So are you running this successfully? Mind sharing your stack and hosting environment?
With fresh Ubuntu 18.04, I just installed some packages to be able to run `make` without issues.
I did exactly the same as stated in the README
Then I use proxychains with a sock tunnel (ssh -D ...) to run whatever I want through the tunnel.
I did exactly the same as stated in the README
t1 IN NS t1ns.mydomain.com. ; note the dot!
t1ns IN A 10.15.213.99
Then I use proxychains with a sock tunnel (ssh -D ...) to run whatever I want through the tunnel.
Check out HomelabOS to help install a bunch of services that can be useful to have on a file server. Many of them are mentioned elsewhere in this thread. It currently has over 50 services you can optionally enable.
The idea is you download it, you run `make`, it asks you a few questions, and then you have a working Dockerized deployment configured via Ansible, with versionable configuration.
https://gitlab.com/NickBusey/HomelabOS
The idea is you download it, you run `make`, it asks you a few questions, and then you have a working Dockerized deployment configured via Ansible, with versionable configuration.
https://gitlab.com/NickBusey/HomelabOS
I used an old VAX for a while connected to full frontal internet with a Telnet port available. One of the cool things VMS had was a way to mirror the session from one terminal on another. It would watch while script kiddies would try to break in, only to be confused about what it was they had broken into when they succeeded.
I'm sure you could do something like this by running VMS on simh and a vlan/pty and use your modem's port routing nat feature to send it telnet traffic.
My go to "spare Linux server" environment has been RasPis for a while though since headless they are pretty cheap, with an actual x86 there are perhaps some interesting PCIe, PCI ,or ISA cards (depends on how old it is) that are fun. Setting it up as a media transfer station to move things off 5.25" floppy or 3.5" floppy or Zip disks or something can be a fun use as well.
I'm sure you could do something like this by running VMS on simh and a vlan/pty and use your modem's port routing nat feature to send it telnet traffic.
My go to "spare Linux server" environment has been RasPis for a while though since headless they are pretty cheap, with an actual x86 there are perhaps some interesting PCIe, PCI ,or ISA cards (depends on how old it is) that are fun. Setting it up as a media transfer station to move things off 5.25" floppy or 3.5" floppy or Zip disks or something can be a fun use as well.
It's a little time consuming to set up correctly and manage, but there is Ampache [1], for creating a scalable streaming music server that can be joined to others, so that catalogs are merged. It was created initially by a bunch of college kids (ok, they were college kids back then) at OSU and paid for with beer.
[1] - https://github.com/ampache/ampache/
[1] - https://github.com/ampache/ampache/
It's really not too bad to setup, I promise! Windows is a pain right now, but Linux is pretty decent. I'm in the process of writing up a guide for setup on Windows for the Wiki, and will also do one for OSX, and a generic one for Linux. The Arch Wiki has a good guide already.
It relies on a webserver, MySQL/MariaDB, and PHP, so a great project to try to get something useful running. Version 4 is going to be released soon and brings a lot of nice changes. I've been trying to update the Wiki to make sure content is up to date and isn't missing important information.
It relies on a webserver, MySQL/MariaDB, and PHP, so a great project to try to get something useful running. Version 4 is going to be released soon and brings a lot of nice changes. I've been trying to update the Wiki to make sure content is up to date and isn't missing important information.
In full disclosure, I only mentioned that because when I had last set it up, it took some tinkering to get it fully functional. That was a few years ago. I'm sure a lot of improvements have been made since then. It probably doesn't help that I tend to harden servers a bit.
No worries, I just wanted to make sure people wouldn't get scared off too easily ;)
This. I have Ampache set up in my cloud server, data storage mounted from my house. Works like a charm.
Also amazing play:Sub iOS app (https://apps.apple.com/us/app/play-sub-music-streamer/id9553...) supports Ampache API and has quite decent UI. All this allows me to have my music anywhere I go
Also amazing play:Sub iOS app (https://apps.apple.com/us/app/play-sub-music-streamer/id9553...) supports Ampache API and has quite decent UI. All this allows me to have my music anywhere I go
NextCloud FTW! Caldav, Carddav, Webdav, dropbox-replacement. What's not to like? Installed in a snap: sudo snap install nextcloud
There was a recent thread with Raspberry Pi ideas that you might find useful: https://news.ycombinator.com/item?id=20264911
A couple things I find useful:
- ZFS fileshare
- Plex Server
- Time Machine server for Macs
- CUPS server, my printer is old and requires drivers, so now I can print from my phone and computers without drivers
- Run your own IRC bouncer if you're into that kind of thing
- Run a seedbox, just for Linux ISOs of course :)
- ZFS fileshare
- Plex Server
- Time Machine server for Macs
- CUPS server, my printer is old and requires drivers, so now I can print from my phone and computers without drivers
- Run your own IRC bouncer if you're into that kind of thing
- Run a seedbox, just for Linux ISOs of course :)
>- Plex Server
I had to stop using plex after they insisted on knowing everything I was watching. Mine runs pihole, a print-server, and a fileserver for kodi now.
I had to stop using plex after they insisted on knowing everything I was watching. Mine runs pihole, a print-server, and a fileserver for kodi now.
My latest project has been to start ripping my entire blu-ray and DVD collection (which is massive), and using Emby to serve them. Since I didn't want my laptop to overheat by compressing the video, I wrote a script to transcode files on my server to a smaller size once the ripping is done.
It's a little different, since my "server" is actually six ODroid XU4's glued together with Docker Swarm and NFS, but it wouldn't be hard to adapt for your thing.
I'd be happy to share that script if you'd like but it's not terribly elaborate.
It's a little different, since my "server" is actually six ODroid XU4's glued together with Docker Swarm and NFS, but it wouldn't be hard to adapt for your thing.
I'd be happy to share that script if you'd like but it's not terribly elaborate.
I'd like to do this someday (Blu-ray format is awful, compared to DVD VOBs).
I was wondering how to make the system physically retain the discs, to be clear that I'm only building a better cache, and the disc still represents a rights physical token. (Not that this is very practical or necessarily necessary, but it's an interesting exercise.)
I was leaning towards seeing how well I could modify the mechanics of an old consumer electronics 400-CD carousel changer, to make it work with whatever Blu-ray drive I found. That would also let me do some other things. A backup plan would be to combine a drive with an additional simple mechanical mechanism, to divert an ejected disc to a stack that is physically retained.
I was wondering how to make the system physically retain the discs, to be clear that I'm only building a better cache, and the disc still represents a rights physical token. (Not that this is very practical or necessarily necessary, but it's an interesting exercise.)
I was leaning towards seeing how well I could modify the mechanics of an old consumer electronics 400-CD carousel changer, to make it work with whatever Blu-ray drive I found. That would also let me do some other things. A backup plan would be to combine a drive with an additional simple mechanical mechanism, to divert an ejected disc to a stack that is physically retained.
What do you use to rip those? I am about to embark on a similar project, planning to use vobcopy so that I can retain all the DVD features (multiple subtitles, multiple soundtracks, menu, etc.), but I suspect there's a better option - I don't want to handbrake as this seems to lose all of those features.
As tombert mentioned, ddrescue to generate an iso is probably fine for DVDs. Note that you will need to use something to initialize the DVD or many drives will just refuse to read out encrypted sectors; something like "dvdbackup -I" will work fine.
As a note, do not use slim dvd or bluray drives for this; I went through several and found that they would have many read errors for DVDs that play fine in actual players and full-height drives.
I am not aware of a good way of getting the full menu experience on bluray discs. For many discs, stripping the menus saves 1-4 minutes of time when watching the movie though, so I haven't worried about it too much.
[edit]
Also, if your DVDs have any dust on them, clean them with a microfiber cloth and spray for cleaning glasses before inserting. ddrescue will usually complete eventually, but it can literally be days with a dirty or scratched disc.
For everyone else: if you don't care about menus &c. makemkv is great. It's not libre software, but the linux version is perpetually in beta so it's free-as-in-beer (though I purchase a license because it's just so darn useful).
As a note, do not use slim dvd or bluray drives for this; I went through several and found that they would have many read errors for DVDs that play fine in actual players and full-height drives.
I am not aware of a good way of getting the full menu experience on bluray discs. For many discs, stripping the menus saves 1-4 minutes of time when watching the movie though, so I haven't worried about it too much.
[edit]
Also, if your DVDs have any dust on them, clean them with a microfiber cloth and spray for cleaning glasses before inserting. ddrescue will usually complete eventually, but it can literally be days with a dirty or scratched disc.
For everyone else: if you don't care about menus &c. makemkv is great. It's not libre software, but the linux version is perpetually in beta so it's free-as-in-beer (though I purchase a license because it's just so darn useful).
I think MakeMKV is slightly better than "free as in beer", because you do have the source to audit, you just can't distribute it willy-nilly like with something with a true FLOSS license.
I too actually bought a license, because it's one of my favorite pieces of software that perpetually "just works".
I too actually bought a license, because it's one of my favorite pieces of software that perpetually "just works".
I thought the bulk of makemkv was closed source, with only a porting layer that is built and linked as part of the install process?
[edit]
I looked at a recent download and a lot more of it delivered as source than I thought (even under a GPL or LGPL in many cases). makemkvcon is shipped as binary only though.
[edit]
I looked at a recent download and a lot more of it delivered as source than I thought (even under a GPL or LGPL in many cases). makemkvcon is shipped as binary only though.
So, my current setup for this is as follows:
1) I plug in a cheapy USB Blu-ray drive plugged into my laptop (Running ubuntu 18.04).
2) I use the software MakeMKV. MakeMKV is open-source-ish; the license is still proprietary, but you can grab the Linux source and compile it yourself off of their forum.
The nice thing about MakeMKV (and the wonderful Matroska format in general) is that it'll retain all the audio tracks and subtitles, which is especially useful for anime. MakeMKV doesn't do transcoding or anything (as far as I know); it just breaks the DRM and puts it into an MKV file.
3) I upload the file to the NFS share of my server
4) I have a script running in my that listens to a folder for new mkv files. If it sees one, it moves it to a working directory, then does `ffmpeg -i $FILE -c:v libx264 -c:a flac -crf 20 ~/processed/rendered_$FILE`
This runs on repeat in the background, and I sometimes have a queue of a few dozen things at a time in that folder (if I'm ripping TV shows or something). As of right now, if one of the nodes crashes, there's no ability to restart and I have to manually move the file back to the queue folder. I've been working on writing a thing with RabbitMQ to fix that, but it's not done yet.
5) Once it's done processing, I manually move the file to the folder hierarchy for Emby, and I typically delete the raw blu-ray source, since those files can be gigantic.
---------
Handbrake is great, and you can tell it to retain multiple tracks and subtitles and whatnot (so it might actually be totally fine for what you're doing), but it's not nearly as versatile as ffmpeg, and as a result I don't really use it for anything more than a quick one-off conversion.
EDIT: Just realized that you said that you want to retain the menus and whatnot...I think you could get away with using something like ddrescue for that.
1) I plug in a cheapy USB Blu-ray drive plugged into my laptop (Running ubuntu 18.04).
2) I use the software MakeMKV. MakeMKV is open-source-ish; the license is still proprietary, but you can grab the Linux source and compile it yourself off of their forum.
The nice thing about MakeMKV (and the wonderful Matroska format in general) is that it'll retain all the audio tracks and subtitles, which is especially useful for anime. MakeMKV doesn't do transcoding or anything (as far as I know); it just breaks the DRM and puts it into an MKV file.
3) I upload the file to the NFS share of my server
4) I have a script running in my that listens to a folder for new mkv files. If it sees one, it moves it to a working directory, then does `ffmpeg -i $FILE -c:v libx264 -c:a flac -crf 20 ~/processed/rendered_$FILE`
This runs on repeat in the background, and I sometimes have a queue of a few dozen things at a time in that folder (if I'm ripping TV shows or something). As of right now, if one of the nodes crashes, there's no ability to restart and I have to manually move the file back to the queue folder. I've been working on writing a thing with RabbitMQ to fix that, but it's not done yet.
5) Once it's done processing, I manually move the file to the folder hierarchy for Emby, and I typically delete the raw blu-ray source, since those files can be gigantic.
---------
Handbrake is great, and you can tell it to retain multiple tracks and subtitles and whatnot (so it might actually be totally fine for what you're doing), but it's not nearly as versatile as ffmpeg, and as a result I don't really use it for anything more than a quick one-off conversion.
EDIT: Just realized that you said that you want to retain the menus and whatnot...I think you could get away with using something like ddrescue for that.
I have a similar setup and I'm curious what your reason for using "c:a flac" over "c:a copy"
Also, I have noticied that for poor DVD transfers crf 20 is insufficient; the result is visibly worse than the original DVD (probably because libx264 is wasting so many bits on preserving the MPEG-2 compression artifacts).
Also, I have noticied that for poor DVD transfers crf 20 is insufficient; the result is visibly worse than the original DVD (probably because libx264 is wasting so many bits on preserving the MPEG-2 compression artifacts).
I use the same script to compress everything, and typically FLAC tends to be a bit more compressed on Blu-ray. I dunno, it seemed like a good idea at the time :)...I should probably do a more-rigorous test to see if it actually decreases the size.
I'm surprised that crf 20 isn't good enough; I admittedly don't own a ton of DVDs nowadays (I bit the bullet and upgraded most things to blu-ray), but on my TV I typically can't tell much any difference after the rip. Granted, none of my DVD transfers have really been "bad"; they're not grainy or gross or anything.
I used to use crf 25 for blu-rays, which worked fine for most things, but in movies that were darker (literally, not tonally), I could see a few compression artifacts, but I don't seem to notice those when I bumped it 20. The ffmpeg documentation itself says that 17 or so should be visually lossless most of the time.
I'm surprised that crf 20 isn't good enough; I admittedly don't own a ton of DVDs nowadays (I bit the bullet and upgraded most things to blu-ray), but on my TV I typically can't tell much any difference after the rip. Granted, none of my DVD transfers have really been "bad"; they're not grainy or gross or anything.
I used to use crf 25 for blu-rays, which worked fine for most things, but in movies that were darker (literally, not tonally), I could see a few compression artifacts, but I don't seem to notice those when I bumped it 20. The ffmpeg documentation itself says that 17 or so should be visually lossless most of the time.
Don't know if it makes a difference, but after checking my history, I was using -crf 21 and bumped it down to -crf 18. My brain is telling me I read somewhere to change cfc in steps of 3, but I'm not sure how reliable that is. So I never tried 20.
I couldn't stop being curious, so I just ripped the blu-ray for "Big Fish", which was 28.9GB raw. After running `ffmpeg -i big_fish.mkv -c:v copy -c:a flac new_big_fish.mkv`, the resulting video was 25.5GB...indicating to me that FLAC compression is worth it, since there's no loss in quality.
FLAC is also just really well-supported pretty much everywhere nowadays, so it's kind of worth it to me regardless.
FLAC is also just really well-supported pretty much everywhere nowadays, so it's kind of worth it to me regardless.
Your test is possibly invalid; without any -map options, ffmpeg will not copy all of the source streams. I don't have big fish, but Les Miserables(2012) I found that flac was about twice the size of the DTS source, which met my expectations:
ffmpeg -i Les\ Misérables\ \(2012\)/title.mkv -map a:1 -c:a copy copy.mkv
ffmpeg -i Les\ Misérables\ \(2012\)/title.mkv -map a:1 -c:a flac flac.mkv
ls -s *.mkv
1749629 copy.mkv 3506729 flac.mkvFair enough point; can't believe I forgot about that.
So do your machines just idle or do you have a way to put them to sleep and only wake/poweron if someone actually wants to watch a movie?
I had a Plex server running on a spare machine, but I found I only used it once a week or so (Friday/Saturday nights) and the rest of the time it was just wasting energy.
I had a Plex server running on a spare machine, but I found I only used it once a week or so (Friday/Saturday nights) and the rest of the time it was just wasting energy.
If you only use Plex infrequently it can be a bit of a power waster for a standalone server. But one of the really cool things about Plex is photo backups from your mobile devices. If you'd rather not have them up in a third party, then it's a really great solution. Also Plex has streaming audio options, though it could be better with audiobooks. My storage array for Plex is in the same place as my home back up so it's never really wasted for me since I store all my files in one location and just open them at whatever computer I'm at.
The ODroids peak at ~20W of power each, and idle at a lot less. All six together, at peak will be ~120W, which isn't so bad, and it seems to idle at around ~40W, which is a number I'm ok with, especially since these things live deep in my basement, far away from the AC, meaning that the heat they generate at least isn't being re-cooled.
That said, I also use these things for a whole ton of things; I watch a lot of movies on there, but since I use docker swarm liberally, pretty much any pet project I have gets deployed there. For example, and I forgot to mention this in the previous post, I built my own basic home-security system by using ffmpeg with a bunch of webcams and Raspberry Pi Zero W's, which communicates to a docker container that streams HLS to hard disk.
Of course I could probably get away with turning these servers on at an as-needed level, but I like being able to just spin up a container whenever I want.
That said, I also use these things for a whole ton of things; I watch a lot of movies on there, but since I use docker swarm liberally, pretty much any pet project I have gets deployed there. For example, and I forgot to mention this in the previous post, I built my own basic home-security system by using ffmpeg with a bunch of webcams and Raspberry Pi Zero W's, which communicates to a docker container that streams HLS to hard disk.
Of course I could probably get away with turning these servers on at an as-needed level, but I like being able to just spin up a container whenever I want.
I like to leave spare machines running chess engines to analyze openings etc. You could also leave it churning for days on end on various Kaggle competitions (especially if it had a relevant GPU). Could also get the thing scraping interesting datasets at a slow and respectful pace.
> especially if it had a relevant GPU
For basic learning on somewhat larger datasets (sub-TB), I have found that my 750 TI SC GPU has been very adequate; the only time I maxed it out was while doing "live training" of a tensorflow CNN from images generated from a driving simulator - so I was using part of the GPU for 3D rendering, and part of it for the training. This was for some "self-driving vehicle" experimentation...
But running the model against the same simulation (that is, having the model drive instead of watching me drive and learning) actually used much less GPU; while I wasn't getting stellar frame rates in either case, they were adequate for the purpose.
I'm just mentioning it, because many people assume you have to have a crazy amount of compute power with a GPU for ML tasks right off the bat (and the associated higher costs of such a system), when in reality that may not be the case (depending on what you are doing and how you are doing it), and you can get away a lesser system.
Where I ran into problems with my GPU, though, was in the case of training using larger datasets (that is, non-streaming) - I couldn't fit them all in the GPU's memory (I think the card only has 1 GB), and so I had to do a batch system - and while it worked well, it was a lot slower because of shuffling data to and from the GPU.
But for my learning purposes, it still did a fine job IMHO; of course, if I were doing such tasks for actual analysis or other "professional" (and paying) purposes, a better card would definitely be in order.
For basic learning on somewhat larger datasets (sub-TB), I have found that my 750 TI SC GPU has been very adequate; the only time I maxed it out was while doing "live training" of a tensorflow CNN from images generated from a driving simulator - so I was using part of the GPU for 3D rendering, and part of it for the training. This was for some "self-driving vehicle" experimentation...
But running the model against the same simulation (that is, having the model drive instead of watching me drive and learning) actually used much less GPU; while I wasn't getting stellar frame rates in either case, they were adequate for the purpose.
I'm just mentioning it, because many people assume you have to have a crazy amount of compute power with a GPU for ML tasks right off the bat (and the associated higher costs of such a system), when in reality that may not be the case (depending on what you are doing and how you are doing it), and you can get away a lesser system.
Where I ran into problems with my GPU, though, was in the case of training using larger datasets (that is, non-streaming) - I couldn't fit them all in the GPU's memory (I think the card only has 1 GB), and so I had to do a batch system - and while it worked well, it was a lot slower because of shuffling data to and from the GPU.
But for my learning purposes, it still did a fine job IMHO; of course, if I were doing such tasks for actual analysis or other "professional" (and paying) purposes, a better card would definitely be in order.
Interested in the chess aspect, what do you do with the analysis when it completes?
Often I'm just looking for some vaguely novel line in an opening that might be interesting to pursue in a game (I am not a strong player but love studying). I sometimes like seeing the difference in analysis between Stockish and Leela of real games (Magnus Carlsen attributes much of his recent form to trying to play more like AlphaZero with all sorts of positional pawn sacrifices etc). I've also tried to do some programmatic stuff around the analyses to automate things (i.e. find a move in a common line that nobody in my database has ever played but the computer says is best, or find lines where it looks like the opponent only has a single path through to stay in the game etc).
If you're interested in learning about Systemd, you could service that runs in the background. Here's an an example of a Slack it I wrote...
A couple years ago & at the last minute, I needed to cover for a sys admin on a new, complicated system that didn't yet have much automation or monitoring. I only got 2 hours of instruction before he needed to disappear for two weeks. We had a status page that showed the status of services, but it wasn't wired to email alerts, PagerDuty, etc.
I really didn't want to keep staring at the status page. So I wrote a simple bash script that curled the status page once per minute. Then the script grep-ed thru the html looking for a line that indicated there was a critical failure. Finally, I did a curl post request to a slack channel that said "the servers are on fire" if there was something to report, and occasionally posted a "the servers are not on fire" if everything was fine. It was a great hack that made life easier while I learned how to properly manage the system, and help with wiring up the monitoring. It was pretty quick to write, too!
A couple years ago & at the last minute, I needed to cover for a sys admin on a new, complicated system that didn't yet have much automation or monitoring. I only got 2 hours of instruction before he needed to disappear for two weeks. We had a status page that showed the status of services, but it wasn't wired to email alerts, PagerDuty, etc.
I really didn't want to keep staring at the status page. So I wrote a simple bash script that curled the status page once per minute. Then the script grep-ed thru the html looking for a line that indicated there was a critical failure. Finally, I did a curl post request to a slack channel that said "the servers are on fire" if there was something to report, and occasionally posted a "the servers are not on fire" if everything was fine. It was a great hack that made life easier while I learned how to properly manage the system, and help with wiring up the monitoring. It was pretty quick to write, too!
I wrote an article about this exact question: https://blog.ivansmirnov.name/what-can-you-do-with-a-persona...
Some new tech that hasn't made it on the list yet:
- https://github.com/issmirnov/zap - recursive URL expander, can be run at DNS level. Ie, "n/" -> "https://news.ycombinator.com/" or "f/g/homelab" -> facebook.com/groups/homelab
- netdata
- https://concourse-ci.org/
- https://varnish-cache.org/intro/
- AFP/NFS/samba for file sharing
- https://github.com/janeczku/calibre-web - web UI for calibre ebooks
- https://jupyter.org/ for python scripts and web scraping
- https://github.com/cdr/code-server for an IDE in a tab.
- https://www.portainer.io/ if you use docker
- https://yourls.org/ for a private URL shortener in your home
Some new tech that hasn't made it on the list yet:
- https://github.com/issmirnov/zap - recursive URL expander, can be run at DNS level. Ie, "n/" -> "https://news.ycombinator.com/" or "f/g/homelab" -> facebook.com/groups/homelab
- netdata
- https://concourse-ci.org/
- https://varnish-cache.org/intro/
- AFP/NFS/samba for file sharing
- https://github.com/janeczku/calibre-web - web UI for calibre ebooks
- https://jupyter.org/ for python scripts and web scraping
- https://github.com/cdr/code-server for an IDE in a tab.
- https://www.portainer.io/ if you use docker
- https://yourls.org/ for a private URL shortener in your home
If you haven't learned Kubernetes yet, it's probably a good use case for that.
Then use any of the other ideas here, except deploy them through Kubernetes.
Then use any of the other ideas here, except deploy them through Kubernetes.
- FreshRSS; get all your RSS feeds remotely in one spot.
- OnlyOffice; this is basically an open-source version of Google Docs. It's rather heavyweight - you will need 2 hosts for this to run and the install uses many varied components, so this is a great opportunity to work with virtualization or containerization.
- EtherCalc; a shared spreadsheet, much easier to setup than OnlyOffice.
- WARCproxy; record your web browsing to a WARC archive and play it back later.
- Install nginx, setup a 64GB VM, install a graphical Linux desktop on it, and use NoVNC and nginx websockets to allow external access to it.
- Play with Yacy, a search engine.
- Seed some Debian torrents and help out the community.
- OnlyOffice; this is basically an open-source version of Google Docs. It's rather heavyweight - you will need 2 hosts for this to run and the install uses many varied components, so this is a great opportunity to work with virtualization or containerization.
- EtherCalc; a shared spreadsheet, much easier to setup than OnlyOffice.
- WARCproxy; record your web browsing to a WARC archive and play it back later.
- Install nginx, setup a 64GB VM, install a graphical Linux desktop on it, and use NoVNC and nginx websockets to allow external access to it.
- Play with Yacy, a search engine.
- Seed some Debian torrents and help out the community.
If its built out of a conventional pc, wipe it and install proxmox on an SSD. With that you can virtualize a openmediavault NAS instance then have spare for spare Linux/windows VM instances, Pihole VM, Nextcloud, etc
If you are close to a window and willing to buy a receiver, install dump1090 and feed to FlightAware and FlightRadar24.
I would sugguest:
1. CUPS print server
2. Pihole
3. Automated Backup storage
4. Network Storage with Samba
At home I use Mozilla Webthings gateway but running on Lubuntu Server instead of the sugguested Raspberry Pi. I even host a local only voice assistant that leverages Gateway to control device states. https://hacks.mozilla.org/2019/04/introducing-mozilla-webthi...
At home I use Mozilla Webthings gateway but running on Lubuntu Server instead of the sugguested Raspberry Pi. I even host a local only voice assistant that leverages Gateway to control device states. https://hacks.mozilla.org/2019/04/introducing-mozilla-webthi...
Gitlab and NextCloud are pretty cool. With NC you can quite easily host your own calendar and contacts.
One of my favorite recent projects I've done was making a full wpa2 wifi access point with firewall out of a Raspberry Pi 3B. (Yes, I know the limitations of hardware. This was a "hold my beer" project.). If your old PC isn't too old and has virtualization instructions, you have a lot of fun with KVM for VMs and iSCSI for a SAN.
I agree with trying to move more of the network functionality out of the ISP provided device and into home servers. A simple and cheap access point[1] can provide all the L1/L2 you need. Then you can run dns and dhcp inside Linux, configure firewall rules, segment your network, etc. The sky is the limit!
Note: I reccomend a dedicated access point over hostapd because I had a lot of problems with it circa 3 years ago.
[1] https://www.dell.com/en-us/work/shop/tp-link-tl-wa801nd-300m...
Note: I reccomend a dedicated access point over hostapd because I had a lot of problems with it circa 3 years ago.
[1] https://www.dell.com/en-us/work/shop/tp-link-tl-wa801nd-300m...
If you plan to keep the server running you could run pihole on it.
I run mine on a raspberry pi zero w. I wouldn’t waste the power of dedicating a whole server to it.
Me neither. My thoughts were: if the server already is running 24/7 for some other reason, you could put pihole on it as well.
You can install bind on it to serve your own dns, or wireguard vpn so you don’t get pwnd in a coffee shop. You could get dnsmasq going so you can give your lan computers proper domain names though it might not play nice with bind if it is on the same server.
Run a freenet node, or setup a YaCy node, or some other network node. An always-on network node for projects like those can really benefit them.
Run a mirror for your favorite projects. This can be for individual programs, or for whole repositories of data like the Debian archive or Wikipedia. Please do check with whatever project you are mirroring if they are OK with it (in case bandwidth will be significant) and what the best practices are that they recommend.
Familiarize with other distributions/operating systems: nixos, one of the bsds, etc.
It's been mentioned in passing form a few times, but if the machine is decent enough (cpu and ram), you might consider setting it up to run and learn Docker or some other virtualization/container system.
Running a Folding@Home client is a helpful way to contribute spare CPU cycles to science...
https://foldingathome.org/
https://foldingathome.org/
If you have tons of space, how about running a blockchain node? Something like FileCoin node is interesting because you can theoretically made some tokens when you’re doing it.
rm -rf /bin
I set up a gitlab ci/cd server. It was fun, just the right about of challenging. Although gitlab has free hosted accounts so maybe not too useful.
Ever since Google Reader was killed I've run a https://tt-rss.org server.
I have my Pi set up with Pi-hole and an IRC bouncer.
Hosting a Tor relay. Also if you contribute to network with at least 500 KBytes/s internet speed, you get a Tor T-shirt after 60 days.
Deploy a single-node kubernetes instance and learn some containers/dev-ops to put on your CV, that'd be my advice.
Don’t expose it to the internet please. Or if you do make sure you know how to secure it first.
Matrix and Mastodon are kinda fun, and federated so you can join a community even if it's just you on your server.
plex, tor relay, bitcoin + lightning nodes
Rosetta@Home - perform medical research using your spare CPU cycles.
Install Asterisk and run your own home PBX using SIP.
Fork bomb. Seriously, it’s a fun learning excersize.
Many years ago I was in a course which had a lab. We would log onto linux machines with network based login(kerberos IIRC). Once logged in, you could also ssh into a server available to us for running codes(limited access).
One day while the lab was in session, I was just playing with stuff and tried executing a fork bomb on the server. The moment I executed the fork bomb, all the 120 machines in the lab became unresponsive, including mine. I didnt realize at that time that they were using the same server for code execution as well as auth server. No one knew what happened and the class was dismissed.
One day while the lab was in session, I was just playing with stuff and tried executing a fork bomb on the server. The moment I executed the fork bomb, all the 120 machines in the lab became unresponsive, including mine. I didnt realize at that time that they were using the same server for code execution as well as auth server. No one knew what happened and the class was dismissed.
A Kubernetes Node.
An elastic search for whatever you fancy searching
An elastic search for whatever you fancy searching
Get to know ZFS.
run a sia node: https://sia.tech/