There is significant work happening in the PostgreSQL ecosystem to make "use a single system to run everything" possible. ParadeDB is one such system pushing on full-text/vector search, with some light aggregations, at the index level. There is also work being done by DuckDB folks with pg_duckdb and others like Xata. (disclaimer: I work at ParadeDB)
ParadeDB maintainer here :). We would happily make it available on Azure (and all other cloud providers!) if there were a way for us to earn a living in doing so.
Fyi, we are in discussion with some hyperscalers on making this possible.
ParadeDB (YC S23, https://paradedb.com) | Database Internals Engineers, Distributed Systems Engineers | Remote or hybrid in San Francisco | Full-time
ParadeDB is a Postgres extension that delivers state-of-the-art search in Postgres. Our customers, including Alibaba Cloud, BILT Rewards, Modern Treasury (YC S18), and many others, rely on us to deliver the performance and feature set of dedicated vector databases/Elasticsearch natively in Postgres, simplifying their stack and day-to-day operations.
If you like systems, this might be for you. We're a team of 10, a Series A company based in SF but distributed across North America. We're hiring for database internals work (e.g., building the database in Rust) and distributed systems work (e.g., building the database-as-a-service platform in Go).
This a 100 times^. The Postgres ecosystem is remarkable and has managed to strike the balance between OSS and commercial successes in a way that most infra verticals have not.
We (ParadeDB) use Barman via CloudNativePG for almost all our deployments. It's been solid, although I've had a few complaints about 1) inability to set S3 storage classes, 2) slow upload for very large databases.
Nonetheless, very happy to see this project on the front page of HN!
ParadeDB (YC S23, https://paradedb.com) | Database Internals Engineers | Remote or hybrid in San Francisco | Full-time
ParadeDB is a Postgres extension that delivers state-of-the-art search in Postgres. Our customers, including Alibaba Cloud, BILT Rewards, Modern Treasury (YC S18) and many others, rely on us to deliver them the performance and feature set of dedicated vector databases/Elasticsearch natively in Postgres, to simplify their stack and day-to-day operations.
If you like systems, this might be for you. We're a team of 11, Series A company based in SF but distributed across North America.
Under the hood, ParadeDB is built by integrating Tantivy (a Lucene-inspired Rust search library) inside Postgres. This is to say: I agree with you -- We're not trying to claim that Postgres itself is an alternative to Lucene, but rather that something like Lucene can be integrated inside Postgres so that you can get the power of both in a single system (or in a cluster of Postgres instances)
(ParadeDB maintainer here) Yes! We've already built some faster analytics in Postgres, and have a lot more coming. Here's some relevant context in case you're curious: https://www.paradedb.com/blog/faceting
One-click deploy with Render, and we're directly in contact with the core team to get it added to their official docs. I hear the PR is up internally :)
(ParadeDB maintainer here). This is super cool. Congrats on the project, and I'm excited to see ParadeDB be used to power this kind of use case. If there's anything else you need to ship Omni, don't hesitate to reach out to me!
That's true. For this reason, most modern search engines support language-aware stemming and tokenization. Popular tokenizers for CJK languages include Lindera and Jieba.
ParadeDB is an alternative to Elasticsearch built on Postgres. We're building a Postgres extension in Rust that offers a new index type optimized for full-text search and aggregate/analytics workloads. We solve three problems with Elasticsearch today:
- Lack of read-after-writes guarantees
- Lack of JOINs
- Infrastructure complexity & cost due to syncing Postgres and Elastic.
We're open-source, and our repository is available at https://github.com/paradedb/paradedb. We're a Series A team of 8 distributed across the US and Canada. Most folks on our team have 10+ years of experience in database internals at companies like Twitter, MongoDB, Oracle, Instacart, etc.
If you know Rust and/or have experience working on DB internals and want to work on cool systems problems, shoot us a note. We hire with conviction, have lots of room to grow, and exciting technical problems to solve. My email is [email protected].
Citus is indeed an example for "distributed PostgreS". There are also serverless Postgres (Neon, Nile, AWS Aurora) which do this.
If you are interested in partitioning in an OLAP scenario, this will soon be coming to pg_analytics, and some other Postgres OLAP providers like Timescale offer it already