It's the kind of software where early venture capital funding can be poisonous rather than helpful. Choosing the right abstractions to build a solid, flexible platform requires a lot of user feedback. You don't get the latter unless you have *paying* customers who have switched their critical processes to your product for a while. You need to build, sell, implement, and provide several upgrades. We bootstrapped a low-code BPM platform (pyrus.com) and were lucky to break even in several years. VCs push you to grow, while premature scaling could be harmful in the long term. It takes time before your platform is mature enough to serve inherently different use cases.
I use pivot tables all the time. The concept is brilliant, but the Excel UI leaves a lot to be desired.
At first, you're amazed at the flexibility, but once you become comfortable, you suddenly hit the limitations. Can't sort by a calculated column, can't categorize without adding columns in the data, etc.
I looked at Quantrix for a while, and it was a bit too complex for practical purposes. I wonder if there are any decent PivotTable tools out there?
We'd use the Record layer, but it was Java-only then. It would require us either to rewrite parts of our backend to Java or to implement some wrappers.
I assume a couple of things here: 1) that SRE costs would be lower with fdb at scale due to its handling outages, i.e. auto-resharding; and 2) that a migration project from *sql to fdb will be finite (hence an investment I hastily called capex).
Would love to hear from anyone with experience in fdb whether these assumptions hold.
We have seriously looked at FoundationDB to replace our SQL-based storage for distributed writes. We decided not to proceed unless we are about to overgrow the existing deploy, a standard leader-follower setup on the off-the-shelf hardware. The limiting factor for the latter would be a number of NMVMe drives we could put into a single machine. It gives us couple dozen Tb of structured data (we don't store blobs in the database) before we have to worry.
fdb is best when your workload is pretty well-defined and will stay such for a decade or so. It is not usually the case for new products which evolve fast. Two most famous installations of fdb are iTunes and Snowflake metadata. When you rewrite petabyte-size database in fdb, you transform continuous SRE/devops opex costs into developers capex investment. It comes with reduced risks for occasional data loss. For me it's mostly a financial decision, not really a technical one.
The article says nothing about how they instantaneously updated millions of user feeds. It was the most challenging task, as it's way easier to scale reads than writes in distributed systems. Rumor has it early Twitter had a target of 5 sec to update everyone of 50M fan feeds when Justin Bieber touched a screen. I would love to hear some technical details on how they did it.
PRQL is a breath of fresh air. Reporting languages generally miss built-in visualization and drill-down capabilities. Ideal reporting query should define not only how to seek, join, and and aggregate data, but also how to visualize output and how to present details in reaction to user clicks. There are some limited efforts like in PowerBI and Splunk but we need a standard. I wonder if PRQL guys will address this need in the future.
https://en.wikipedia.org/wiki/Snowflake_ID
https://github.com/sony/sonyflake?tab=readme-ov-file