As a library maintainer, skill and taste are almost equally important. If I can’t recognize inefficiencies, difficult to maintain code, or generally unpleasant code smells, then people lose trust in my libraries/products and it’s no better off than some recently generated slop.
Years of production experience, wisdom, and using something in anger matters for both skill and taste.
The efforts we've undergone to make Oban (and Pro) work with CRDB have been ridiculous. Feature detection all over because of a lack of common operators and functions that can't be used in indexes. The worst is the rampant "serialization_failure" errors that force continual transaction retries. Not how I'd suggest scaling Postgres.
That said, as a predecessor to dbos in building durable workflows just using Postgres, I concur with the overall sentiment.
Nearly this, but using ghostty instead of tmux. You don’t get the remote connection aspect of tmux, but for splitting/zooming/preserving windows it is fantastic. The best part is you can configure natural shortcuts rather than using a leader for everything.
With a typical Redis or RabbitMQ backed durable queue you’re not guaranteed to get the job back at all after an unexpected shutdown. That quote is also a little incorrect—producer liveness is tracked the same way, it’s purely how “orphaned” jobs are rescued that is different.
You can have jobs that run as long as you like. The difference is purely in how quickly they are restored after a crash or a shutdown that doesn’t wait long enough.
This is absolutely true (except we went OSS + Web initially, Pro came later). You were an inspiration, always helpful in discussion, and definitely paved the way for this business model.
> The vast, vast majority of Python libraries are not async-friendly and most still rely on the GIL. On the other hand, Celery has absolutely no asyncio support at all, which sets the pro feature apart.
That's great advice. Wish we'd been in contact before =)
Transactions around fetching/updating aren't trivial, that's true. However, the work that you're doing _is_ regular activity because it's part of your application logic. That's data about the state of your overall system and it is extremely helpful for it to stay with the app (not to mention how nice it makes testing).
Regarding overall throughput, we've written about running one million jobs a minute [1] on a single queue, and there are numerous companies running hundreds of millions of jobs a day with oban/postgres.
There are other projects that implement the ideas in OSS, but that's the same in Elixir. Not that we necessarily invented DAGs/workflows, but our durable implementation on the Elixir side predates DBOS by several years. We've considered it an add-on to what Oban offers, rather than the entire product.
Having an entirely open source offering and selling support would be an absolute dream. Maybe we'll get there too.
> It supports workflows, rate limiting, unique jobs, bulk operations, transactional enqueuing, etc. Why not move these things to the OSS version to be competitive with existing options, and focus on dedicated support and more traditional "enterprise" features, which absolutely are worth $135/month (the Oban devs provide world-class support for issues).
We may well move some of those things to the OSS version, depending on interest, usage, etc. It's much easier to make things free than the other way around. Some Pro only features in Elixir have moved to OSS previously, and as a result of this project some additional functionality will also be moved.
Support only options aren't going to cut it in our experience; but maybe that'll be different with Python.
> There are many more options available in the Python ecosystem than Elixir, so you're competing against Temporal, Trigger, Prefect, Dagster, Airflow, etc etc.
There's a lot more of everything available in the Python ecosystem =)
Pleased to see this posted! A lot of design time and effort behind this project (something we'll be speaking about this year). Happy to answer any questions people may have.
Years of production experience, wisdom, and using something in anger matters for both skill and taste.