Show HN: FastScheduler – Decorator-first Python task scheduler, async support(github.com)
github.com
Show HN: FastScheduler – Decorator-first Python task scheduler, async support
https://github.com/MichielMe/fastscheduler
14 comments
If Celery seems like overkill for your process, and you're really just looking to execute basic cron functioanlity, then why not just use crontab to invoke your Python script?
I can think of two major ways to operationalize a Python script that needs to run continuously. One is with containerization, which usually means Kubernetes, which already has a perfectly fine resource definition for cronjobs. The other approach is to run the script in a bare metal or VM, which would mean defining a service to ensure that the process can be managed appropriately, restarted if it dies, and the like. In other words, defining a service is about just as much effort as defining a cronjob, and there's no escape from some amount of "ops work" that isn't encapsulated in a Python script.
So why not just use the tried-and-true prior art than worry about building and supporting your own secret third thing that others would need to learn, support, maintain, and keep in mind when debugging a problem?
I can think of two major ways to operationalize a Python script that needs to run continuously. One is with containerization, which usually means Kubernetes, which already has a perfectly fine resource definition for cronjobs. The other approach is to run the script in a bare metal or VM, which would mean defining a service to ensure that the process can be managed appropriately, restarted if it dies, and the like. In other words, defining a service is about just as much effort as defining a cronjob, and there's no escape from some amount of "ops work" that isn't encapsulated in a Python script.
So why not just use the tried-and-true prior art than worry about building and supporting your own secret third thing that others would need to learn, support, maintain, and keep in mind when debugging a problem?
Fair point. Cron works fine for standalone scripts. This is more for when you want scheduled tasks inside an existing Python app without spinning up separate infrastructure.
Looks interesting. Wondering how this is different from the more established https://github.com/agronholm/apscheduler ?
APScheduler is solid and more mature. Main difference is the API — FastScheduler is decorator-first so you get @scheduler.daily.at("09:00") instead of configuring triggers, executors, and job stores separately. Also has a built-in FastAPI dashboard.
Maybe I forgot about this, but how would you secure the scheduler api/dashboard from unauthorized access? This might be a good point to add to the readme.
Good point. Right now it relies on FastAPI's dependency injection — you can wrap the router with your own auth middleware or add dependencies when including it. But I should add an example to the docs. Thanks for the nudge.
This is really cool, and I could see myself using this. Sometimes I need functionality like this, but can't be bothered to build up the infrastructure around it. This is perfect for that use case.
Thanks! Yeah that's exactly the use case — when you just need something scheduled without setting up a whole stack.
This looks great OP. Do you have anything on the roadmap that you’d be open to receiving PRs for? I noticed there weren’t any issues in the repo and would be keen to lend a hand!
Definitely open to PRs! Still early days so lots of room for improvement. Feel free to open an issue if you have ideas.
I would deploy this today to run my backups if jobs could be defined in the UI as well.
Interesting idea. Right now jobs are code-only which keeps it simple, but a UI for defining basic jobs could work. I'll think about it.
FYI I had a similar problem to yours in terms of having jobs I wanted to run on a schedule. I also had an extra layer where I wanted to let users to define jobs with their own special parameters etc. Maybe what I did is helpful for you:
- Form submission in front-end admin panel for users, for "new scheduled job"
- Form allows defining a job name, job type (while I let users define jobs, I limit it to a subset of python functions that I trust but are still general enough), job parameters (just a json blob for kwargs for the python func), frequency, and timeout.
- For the whitelisting of functions it's easiest to have a directory in your src which has the different python functions that are usable, and when you do a "get" for the form you populate the dropdown with the scripts from that directory / some metadata.
- The backend (fastAPI) saves the details in a DB and creates a CRON file for the job, to add to the crontab. I basically have a template bash statement with the timeout built in and logging hooked up to it. And when python functions are called I have a helper which grabs the JSON kwargs from the DB or a filecache (if remote DB and I don't want to hit it every minute or something) to avoid cmd line injection.
- Also have an "edit scheduled job" which opens the same form view but with items pre-populated, and the submit going to a patch endpoint instead of a post.
It's stupid simple but lets users define a range of jobs. Things like having daily backups, where to send the backups, pulling things from / pushing things to an API frequently, etc.
- Form submission in front-end admin panel for users, for "new scheduled job"
- Form allows defining a job name, job type (while I let users define jobs, I limit it to a subset of python functions that I trust but are still general enough), job parameters (just a json blob for kwargs for the python func), frequency, and timeout.
- For the whitelisting of functions it's easiest to have a directory in your src which has the different python functions that are usable, and when you do a "get" for the form you populate the dropdown with the scripts from that directory / some metadata.
- The backend (fastAPI) saves the details in a DB and creates a CRON file for the job, to add to the crontab. I basically have a template bash statement with the timeout built in and logging hooked up to it. And when python functions are called I have a helper which grabs the JSON kwargs from the DB or a filecache (if remote DB and I don't want to hit it every minute or something) to avoid cmd line injection.
- Also have an "edit scheduled job" which opens the same form view but with items pre-populated, and the submit going to a patch endpoint instead of a post.
It's stupid simple but lets users define a range of jobs. Things like having daily backups, where to send the backups, pulling things from / pushing things to an API frequently, etc.
Really appreciate you sharing this setup. The cron file generation for persistence is clever, and the whitelisting approach for user-defined jobs makes a lot of sense. Might borrow some of these ideas if I add a UI for job creation. Thanks!
So it's decorators for scheduling (@scheduler.every(5).minutes, @scheduler.daily.at("09:00")), state saves to JSON so jobs survive restarts, and there's an optional FastAPI dashboard if you want to see what's running.
No Redis, no message broker, runs in-process with your app. Trade-off is it's single process only — if you need distributed workers, stick with Celery.