Making the workflow engine of DBOS depend on the paid component (Conductor) for scaling and recovery makes it a no-go. River also has "traps" like not supporting DLQ, which is a paid feature.
DBOS looks simple (good), but from the docs below, executor elasticity appears to be locked behind license purchase. So it truly is like docker compose, good and bad parts?
>When self-hosting in a distributed setting without Conductor, it is important to manage workflow recovery so that when an executor crashes, restarts, or is shut down, its workflows are recovered. You should assign each executor running a DBOS application an executor ID through DBOS configuration. Each workflow is tagged with the ID of the executor that started it. When an application with an executor ID restarts, it only recovers pending workflows assigned to that executor ID.
Well, in my early days programming python I made a lot(!!) of code assuming non-concurrent execution, but some of that code will break in the future with GIL removal. Hopefully the Python devs keep these important changes as opt-ins.
I expect to have at least 15 more years in the workforce and I hate that I have to live through this "revolution". I worry about what will be final balance of lives improved vs lives worsened.
Congrats on the progress!
What is the behavior of PgDoc if it receives some sort of query it can't currently handle properly? Is there a linter/static analysis tool I can use to evaluate if my query will work?
The distinction is more clear when indexing actual text and applying tokenization. A "typical" index on a database column goes like "column(value => rows)". When people mention inverted indexes its usually in the context of full text search, where "column value" usually goes through tokenization and you build an index for all N tokens of a column "column:(token 1 => rows)", "column:(token 2 => rows)",... "column:(token N => rows)".
Not the person you have asked but at work (we are a CRM platform) we allow our clients to arbitrarily query their userbase to find matching users for marketing campaigns (email, sms, whatsapp). These campaigns can some times target a few hundred thousand people. We are on a really ancient version of ES, but it sucks at this job in terms of throughput. Some experimenting with bigquery indicates it is so much better at mass exporting.