Ask HN: What scripts/cronjobs you have made that made your life generally easier
Could be work or life related - trying to get inspiration to write scripts/cronjobs for my own life.
45 comments
I used to back up my friend's website over FTP to a git repo. He often would break something and have no way to know what broke or how to fix it.
I frequently set up automatic backups of network appliance configurations, route tables, and traceroute results. This is useful for diagnosing sudden problems, as well as taking before and after snapshots when making changes.
I collect interesting data, such as kernel version and versions of various services, from hosted web servers my clients run.
I collect data at regular intervals when investigating a sporadic, chronic trouble event to help me identify correlations and test hypotheses.
I collect BGP and traceroute data at regular intervals from route servers to identify important changes and trends in Internet routing for specific prefixes.
I used to collect IDS signatures in a git repo to track updates, but I eventually moved this to GitLab.
I regularly run a watchdog script to check for, alert on, and attempt to mitigate low disk space and other such conditions.
Hope this helps!
I frequently set up automatic backups of network appliance configurations, route tables, and traceroute results. This is useful for diagnosing sudden problems, as well as taking before and after snapshots when making changes.
I collect interesting data, such as kernel version and versions of various services, from hosted web servers my clients run.
I collect data at regular intervals when investigating a sporadic, chronic trouble event to help me identify correlations and test hypotheses.
I collect BGP and traceroute data at regular intervals from route servers to identify important changes and trends in Internet routing for specific prefixes.
I used to collect IDS signatures in a git repo to track updates, but I eventually moved this to GitLab.
I regularly run a watchdog script to check for, alert on, and attempt to mitigate low disk space and other such conditions.
Hope this helps!
I listen to the same radio station on my commute to/from work, and to avoid needing to use Shazam etc., I wrote a script which parses the radio station's "now playing" web page and logs it to a file every 2 minutes (if the song is different from at the last run). Then I can more easily find/remember songs I liked and see trends etc.
No clue if it's still active but I think that Last.fm can automate that.
http://scrobblealong.com
http://scrobblealong.com
Genius!
1. A script that opens all of my morning apps and browser tabs in one command.
2. I make a run.sh script in every project that I add to my git exclude, so whether it's a Go app, Python app, Node app, makefiles, Docker, ..., no matter what I can go into any project and run this script to start it. Then I add an alias in my bash profile so I just type "r" to ./run.sh. For more complicated apps, I generalize this to have a build.sh, deploy.sh, etc.
2. I make a run.sh script in every project that I add to my git exclude, so whether it's a Go app, Python app, Node app, makefiles, Docker, ..., no matter what I can go into any project and run this script to start it. Then I add an alias in my bash profile so I just type "r" to ./run.sh. For more complicated apps, I generalize this to have a build.sh, deploy.sh, etc.
... maybe this is crazy, but why not just name the script r and do away with the alias? that way your shell can let you know "hey that's not an executable in this dir" if you ever try and run it somewhere it isn't instead of "hell i'll run that alias for you... oh "
The argument I have for keeping the .sh convention is to be able to go into an arbitrary project and run `ls *.sh` to see all of my commands.
Ooh, sounds like an opportunity to alias `ls *.sh` to something handy to list your available commands. You could even format it pretty.
I wonder if you could further play around by having your command aliases default to figuring out what sort of project you're in and either doing the nearest 'generic' method of completing that command for that type of project or perhaps even generating a skeleton file and opening up your $EDITOR to edit to fit the project you're in.
I wonder if you could further play around by having your command aliases default to figuring out what sort of project you're in and either doing the nearest 'generic' method of completing that command for that type of project or perhaps even generating a skeleton file and opening up your $EDITOR to edit to fit the project you're in.
I do something similar, but with make.
I wanted to go this route but it felt weird in projects that already had makefiles committed in the repo used by others to have to name my Makefile something else and then run like:
make -f Makefile2 build ...
I've never updated a repo with a make file and I needed to script things that didn't belong in the main make file. I'm not sure what I'd prefer to do.
I have rewritten the BSD calendar app in every language I can get my hands on, from Perl to Go to Emacs Lisp to.... It isn't that I want a calendar app, but it has helped me to develop strong fluency in porting code from one language to another, both at work and in my side projects.
I like the BSD calendar program, but on NetBSD, it doesn't work unless you install the compiler set. I also wish it supported the ISO 8601 date format.
Have you tried porting to Perl 6 yet? https://perl6.org
I used to host a bunch of Wordpress and Drupal sites on Linux servers. Properly setting file and directory permissions is a huge part of keeping those CMSs secure. The defaults are bad and there is a ton of horrible advice online in the form of troubleshooting threads with responses like "I made the error go away by setting everything to 777!"
So I wrote two shell scripts (one for each CMS) that could be easily customized for each site. I ran it by hand whenever we did a deployment, and also set it to run every hour in crontab. That way even if someone changed the permissions away from safe (by accident or maliciously), they would revert to safe pretty quickly.
I've since moved all those sites to WP Engine and Pantheon, so now the directory permissions are someone else's problem. :-)
So I wrote two shell scripts (one for each CMS) that could be easily customized for each site. I ran it by hand whenever we did a deployment, and also set it to run every hour in crontab. That way even if someone changed the permissions away from safe (by accident or maliciously), they would revert to safe pretty quickly.
I've since moved all those sites to WP Engine and Pantheon, so now the directory permissions are someone else's problem. :-)
I'm starting a tiny personal wp site that I'm self hosting and would love the collected wisdom in those scripts if they're still around.
The one that "switches off" all internet access from 08h00 to 14h00 has done wonders for my productivity (in a positive way!)
Love my little tasker app that lets me pick how long to surf for and blocks the screen after that many minutes.
Finally - a script that also includes a positive life impact!
Cron job every two mintues on my raspberry pi to switch a relay on/off depending on the temperature in my shed.
Relay is attached to a 500w heater and 4 old computer fans blowing the hot air at my feet.
Relay is attached to a 500w heater and 4 old computer fans blowing the hot air at my feet.
My work uses google calendars but the calendar reminders aren't enough for me to make meetings (and I don't look at my phone for calendar alerts), so I use cron combined with gcalcli (https://github.com/insanum/gcalcli) to give me desktop popups using kdialog for meeting reminders. One at 15 minutes, one at 10 minutes, one at 5 minutes, and one at 1 minute. I now make most meetings!
/usr/bin/gcalcli --calendar '[my calendar]' --detail_location remind 10 '/path/to/alertscript.sh "%s"'
where alertscript.sh just calls kdialog with the message describing the meeting.
/usr/bin/gcalcli --calendar '[my calendar]' --detail_location remind 10 '/path/to/alertscript.sh "%s"'
where alertscript.sh just calls kdialog with the message describing the meeting.
A couple of life-related ones that I regularly use:
bank csv to sqlite: stores all of my banking transactions in sqlite.
high-tide-calendar-beach: Uses XTide to create a csv of high/low tide times for a local beach and then converts it to an ics calendar that gets copied to a server where my calendar apps read it. (The local beach is inaccessible at high hide)
bank csv to sqlite: stores all of my banking transactions in sqlite.
high-tide-calendar-beach: Uses XTide to create a csv of high/low tide times for a local beach and then converts it to an ics calendar that gets copied to a server where my calendar apps read it. (The local beach is inaccessible at high hide)
https://flaterco.com/xtide/
XTide is a pretty cool concept. So, I'm linking it here.
XTide is a pretty cool concept. So, I'm linking it here.
I'm interested in the bank csv to sqlite one, any chance of sharing it?
I have an fcron[1] job that will perform a mysqldump, encrypt the resultant file using an incremental file backup program called borg[2], rclone the same to a remote site and log the time of the last run to a text file. I do the same for my dotfiles too, which is collected inside a single folder and make use of stow[3]
[1] http://fcron.free.fr/
[2] https://borgbackup.readthedocs.io/en/stable/
[3] https://www.gnu.org/software/stow/manual/stow.html
[1] http://fcron.free.fr/
[2] https://borgbackup.readthedocs.io/en/stable/
[3] https://www.gnu.org/software/stow/manual/stow.html
The best ones have been auto complete scripts for more mundane ones like in this tutorial: https://iridakos.com/tutorials/2018/03/01/bash-programmable-... .
One example I deal with often is interacting with third party systems. I had scripts to simulate them for ages but had to copy the values to generate the correct files from the UI/database. Now the auto-complete scripts look up the values in the database directly so I can usually just use tab completion.
One example I deal with often is interacting with third party systems. I had scripts to simulate them for ages but had to copy the values to generate the correct files from the UI/database. Now the auto-complete scripts look up the values in the database directly so I can usually just use tab completion.
I love writing shell/bash scripts to automate stuff on our servers. Some little things I have done:
- Wrote a script to take automatic backups from server to AWS S3 bucket and add that script to a cron.
- Wrote an all-in-one script to install/configure/setup WordPress on a barebone VPS server using a LEMP stack along with LetsEncrypt SSL. Fully automated but you need to have a domain pointing to the server for letsencrypt. Once you have a basic Ubuntu server provisioned, this script takes care of everything else.
- Playing with a simple script that sends basic server stats to an API endpoint (I know I could use monitoring tools but where is the fun in that :))
- Wrote a script to take automatic backups from server to AWS S3 bucket and add that script to a cron.
- Wrote an all-in-one script to install/configure/setup WordPress on a barebone VPS server using a LEMP stack along with LetsEncrypt SSL. Fully automated but you need to have a domain pointing to the server for letsencrypt. Once you have a basic Ubuntu server provisioned, this script takes care of everything else.
- Playing with a simple script that sends basic server stats to an API endpoint (I know I could use monitoring tools but where is the fun in that :))
A tiny script in my google account that regularly scans the next week and auto-rejects meetings invites when they don't have an agenda and a goal specified. It then emails the meeting owner with a request to change.
PM me for the code.
PM me for the code.
Is PM'ing a feature here I don't know about? Looked at your profile and I don't see an email address. Would love to see the code. Sounds crazy useful.
Oops yeah I assumed.
Here is the gist https://gist.github.com/christophstrasen/65461f921b5e8a8be23...
Here is the gist https://gist.github.com/christophstrasen/65461f921b5e8a8be23...
Auto purging my downloads folder with a scheduled task.
I received a new laptop for work this spring and was determined to make my downloads folder strictly temp storage, but loathed putting any sort of manual work into it periodically.
I have a small powershell script which moves all files in downloads older than X days into a 'dumpster' folder (not straight to recycle bin), and then all dumpster files older than Y days, into the recycle bin.
Scheduled in the task manager to run daily. And for the first time, I have a tame downloads folder.
I received a new laptop for work this spring and was determined to make my downloads folder strictly temp storage, but loathed putting any sort of manual work into it periodically.
I have a small powershell script which moves all files in downloads older than X days into a 'dumpster' folder (not straight to recycle bin), and then all dumpster files older than Y days, into the recycle bin.
Scheduled in the task manager to run daily. And for the first time, I have a tame downloads folder.
browser script: one that shows me duckduckgo "bangs" for any site I'm on (search keywords that forward directly to a 3rd party search)
browser script for work (web stuff): auto fill for creating test accounts, indicating which server I'm on, injecting little shortcut links here and there.
With help the windows utility shareX I can paste markdown via a shortcut and get a sharable link with html on my server back in to the clipboard.
not mine but great: I saw someone having readmes and todos in project folders and than aggregating them
browser script for work (web stuff): auto fill for creating test accounts, indicating which server I'm on, injecting little shortcut links here and there.
With help the windows utility shareX I can paste markdown via a shortcut and get a sharable link with html on my server back in to the clipboard.
not mine but great: I saw someone having readmes and todos in project folders and than aggregating them
I've created a script that installs all necessary stuff for me if I would need to reinstall my system - saved it on remote repo to just download & run it after fresh system is installed: https://github.com/lukaszkups/linux-installation-script (it might be now pretty outdated)
I wrote a cron job to send me weekly email digest of things I learn. During the week days, I use my open source program [0] to quickly write down things as I learn them. On Friday, I receive a email digest to refresh my memory.
This method put spaced repetition on an autopilot and helped me learn more.
[0] - https://github.com/dnote/cli
This method put spaced repetition on an autopilot and helped me learn more.
[0] - https://github.com/dnote/cli
I used my bash WebDriver tool (https://github.com/mbrock/wd) to write a script that logs into my online bank in a browser window, prompting me in the terminal for the one-time code from the little gadget, because for some reason I find clicking around in the bank website so truly horrible.
But doesn't that mean your credentials are saved in plaintext (in the script)? I guess the 2FA makes that a little less dangerous...
The 2FA makes it secure as far as I'm concerned... I'm not all that paranoid. (If you hack my computer, you steal my first authentication factor anyway...)
AutoHotKey is the big help here. Shortcuts instead of hunting for files, shortcuts instead of repetitive menu commands, remapping keys to do screenshots or launch KDiff or whatever. Like for example, I set one up that only applies to Outlook and applies a monospace style to the selected text so it looks like code. That was a bunch of clicks before and now it's just Ctrl+Q.
every time my dynamic ips change (work machines), I send a message with the latest ips to my slack, so that when I vpn+ssh, I know the ips.
I made a pseudo-grandfather clock with an RPi connected to a $1 motor controller + servo, cronjobs call a little python script that moves the servo to strike a long chime N times on the hour (and I have it scheduled to ring during times of the day when I am usually home and awake, and otherwise keep quiet).
- Periodically delete postgres table records older than (x) days and also, move the records older than (y) days to cold storage.
- Script/Bot, to download my favourite songs searching from multiple sources given its name as input.
- Script/Bot, to download my favourite songs searching from multiple sources given its name as input.
In android terminal, I made a script which would set a nice coloured PS1, a few handy aliases, shell functions & PATH extensions upon app startup.
BTW command line on android is very useful.
A script sending an IFTTT push message when a torrent download starts or finished downloading
cleanup psql logs older than (x) amt of days on production instances