HackerTrans
TopNewTrendsCommentsPastAskShowJobs

iamlintaoz

no profile record

Submissions

Don't Split My Data: I Will Use a Database (Not PostgreSQL) for My Data Needs

eloqdata.com
1 points·by iamlintaoz·há 8 meses·1 comments

Show HN: EloqDoc: MongoDB-compatible doc DB with object storage as first citizen

github.com
46 points·by iamlintaoz·há 9 meses·18 comments

comments

iamlintaoz
·há 5 meses·discuss
This is not true at all. China’s education system is nationally standardized. Although economic development is uneven with far greater investment concentrated in major cities than inland regions, the structure of education itself is consistent across the country. Schools follow the same national curriculum and use the same core teaching materials.

Income disparities may have some impact on teacher quality, but the difference is often less significant than people assume. Broad access to education tends to matter more than whether a particular middle-school teacher is exceptional. In fact, students in some inland provinces frequently achieve very high scores on the national college entrance examination, driven in part by strong incentives to gain admission to top universities and pursue opportunities in more economically developed regions.

Among younger generations, illiteracy is virtually nonexistent. With nine years of compulsory education mandated nationwide, basic literacy rates are effectively at 100 percent.
iamlintaoz
·há 5 meses·discuss
It’s true that learning Chinese as an adult—especially if you come from an English or other European language background—can be extremely challenging. I have several colleagues who have lived in Beijing for more than a decade, are married to Chinese spouses, and still can barely speak the language, it becomes even more challenging for reading.

This creates real difficulties in daily life. Today, almost all routine activities—online shopping, digital payments, banking, ride-hailing—are conducted through smartphone apps. If you can’t read Chinese, even basic tasks become complicated. In recent years, the number of foreigners living in China has declined compared to a decade ago. While political and economic factors clearly play a role, I suspect that the language barrier has also become a more significant obstacle.

Many Chinese people, especially younger generations, can speak some basic English, since it is a mandatory subject in school. As a result, interpersonal communication is usually manageable, and traveling in China is relatively easy. However, living there long-term is a very different experience from visiting as a tourist.
iamlintaoz
·há 6 meses·discuss
The disk storage EloqKV uses (EloqStore [1]) is optimized for batch updates because the upper Data Substrate layer manages buffering and the Write-Ahead Log (WAL), absorbing writes and guaranteeing durability. When durability is not required, the WAL can be optionally disabled.

[1] github.com/eloqdata/eloqstore

Disclaimer: I am the CEO of EloqData
iamlintaoz
·há 6 meses·discuss
If you need so many tricks to support the infra, it will eventually come back to bite you. I am pretty sure that Google in year 2000 could have supported their workloads with existing technologies (Yahoo could, and it was a much larger company). But they did GFS and Bigtable, and the rest is history. Other companies struggled to catch up due to inferior infrastructure. A visionary company needs to be prepared and should not be hindered by infrastructure. Can you scale the single primary system another 10x or more? Because their CEO said that they will scale their revenue by that much within just a couple of years.
iamlintaoz
·há 9 meses·discuss
These are great questions. I appreciate you carefully reading through the documents. For the first question, we have detailed benchmark on EloqKV, with the same architecture (but with Redis API) in our blog, and we will soon publish more about the performance characteristics of EloqDoc. Overall, we achieve about the same performance as using local NVME SSD, even when we use S3 as the primary storage, and the performance often exceed the original database implementation (in the case of EloqDoc, original MongoDB).

As for the durability part, our key innovations is to split state into 3 parts: in memory, in WAL, and in data storage. We use a small EBS volume for WAL, and storage is in S3. So, durability is guaranteed by [Storage AND (WAL OR Mem)). Unless Storage (S3) fails, or Both WAL (i.e. EBS lost) AND Mem fail (i.e. node crash), persistence is guaranteed. You can see the explanation in [1]

[1] https://www.eloqdata.com/blog/2025/07/16/data-substrate-bene...
iamlintaoz
·há 9 meses·discuss
That's exactly the reason. S3 is better in almost all aspects compared with EBS, except the performance part, and I am glad that our Data Substrate technology solved this issue gracefully [1].

As for the compatibility, we are leveraging some of the code from 4.03 version (the last AGPL version), and we have a very good compatibility (we will show some results in later blog posts). As I mentioned in another reply post, the Mongo APIs are reasonably stable over the last few years, only seeing very minor changes. Most of the later versions improved upon performance and transaction supports, which we support natively with our underlying data substrate technologies. Still, if you have any specific API that you feel is needed, we'd be happy to implement and we welcome community contributions.

Multi-master/multi-writer means it is a fully distributed database. Of course you can run it in single node configurations and get all the single node benefits, but if deployed in a cluster, you do not need to worry about which node to write to, or how data are sharded. If you writes potentially can cause conflicts (i.e. write to the same data at the same time on different nodes), the concurrency-control will handle that for you. In fact, you will encounter the same issue even in a single node configuration, since a single node is still multi-threaded.

[1] https://www.eloqdata.com/blog/2025/07/16/data-substrate-bene...
iamlintaoz
·há 9 meses·discuss
For full transparency, I am the CEO of EloqData and I am happy to answer any questions you have. EloqDoc has been open-sourced for awhile, and we are glad to announce that the current code is stable for you to test on your workload.
iamlintaoz
·há 9 meses·discuss
Thank you so much for reporting this! My local testing on Chrome privacy mode was fine, but we will definitely prioritize checking and fixing the issue in Firefox's Strict Privacy Mode.
iamlintaoz
·há 9 meses·discuss
That is right. We leverage some of the AGPL MongoDB code for the parser, as indicated by the License. Our own code can be licensed differently, see a previous discussion on Hacker News [1]. The Mongo API are reasonably stable over the last few years, only seeing very minor changes. Most of the later versions improved upon performance and transactions, which we support natively with our underlying technologies. Still, if you have any specific API that you feel is needed, we'd be happy to implement and we welcome community contributions.

[1] https://news.ycombinator.com/item?id=44937978
iamlintaoz
·há 9 meses·discuss
That’s correct. FerretDB and DocumentDB both pursue PostgreSQL-based document models.

EloqDoc follows a fundamentally different architectural path. Instead of executing in Postgres, we leverage the existing MongoDB parser and executor to ensure maximum compatibility. Our main contribution is to replace the single-writer WiredTiger storage engine entirely with Data Substrate.

The Data Substrate is an abstract layer built specifically to handle distributed database fundamentals: scalable buffer pooling, concurrency control, durability, elasticity, and fault tolerance. You can read more: https://www.eloqdata.com/blog/2025/07/14/technology

This architectural choice is what enables EloqDoc to deliver features that Postgres-based solutions cannot easily match, including: full compatibility, native Multi-Writer capability, Object Storage First and extremely low-latency distributed transactions.
iamlintaoz
·há 9 meses·discuss
Yes, this is exactly the typical use case where EloqDoc shines.

Our auto-tiering feature allows you to manage a massive database (e.g., 5TB+) primarily on cheap, durable object storage (like S3), which is already built for cross-AZ replication and is up to 3x cheaper than traditional cloud block storage (EBS).

We use local NVMe SSDs (200K+ IOPS) purely as a high-performance cache layer to accelerate read access. Hot and cold data are automatically swapped across memory, SSD, and object storage tiers based on access frequency, ensuring high performance (up to 100x faster than archive solutions)