This trend has been obvious since at least the Poland deal. Korea gets much more return on its defense dollar manufacturing exportable weapons systems than relying on imports or domestic-only programs.
This has a lot in common with a columnstore, but there are a few interesting differences.
The main one is subrow versioning. Column stores (in OLAP at least) have always used row-level versioning, which gets in the way of small updates. A single change to a row amplifies into deleting and re-inserting the whole thing, and operations that seem sensible like adding or dropping a column break previous versions. This scheme is the first I've seen that tries to fix that problem.
One other difference is a lack of compression, as it's zero-copy, so the performance gains of operating on compressed data are lost.
One of the founders of Antithesis gave a talk about this problem last week; diversity in test cases is definitely an issue they're trying to tackle. The example he gave was Spanner tests not filling its cache due to jittering near zero under random inputs. Not doing that appears to be a company goal.
This site is underweighted on OLAP. Columnstores were invented for precisely this use case; nobody in the field wants to normalize everything.
Which brings me to the question, why a rowstore? Are Z-sets hard to manage otherwise?
Another aspect of wide tables is that they tend to have a lot of dependencies, ie different columns come from different aggregations, and the whole table gets held up if one of them is late. IVM seems like a good solution for that problem.
He doesn't want to manage the database the way he manages the rest of his infrastructure. All of his bullet points apply to other components as well, but he's absorbed the cost of managing them and assigning responsibilities.
- Crud accumulates in the [infrastructure thingie], and it’s unclear if it can be deleted.
- When there are performance issues, infrastructure (without deep product knowledge) has to debug the [infrastructure thingie] and figure out who to redirect to
- [infrastructure thingie] users can push bad code that does bad things to the [infrastructure thingie]. These bad things may PagerDuty alert the infrastructure team (since they own the [infrastructure thingie]). It feels bad to wake up one team for another team’s issue. With application owned [infrastructure thingies], the application team is the first responder.
The object storage stuff is new, but it's mostly confirmed that the older architecture works. MPP with shared (S3) storage and everything above that on local SSD and compute delivers the best performance. Even Snowflake finally came out with "interactive" warehouses with this architecture.
Parquet, Iceberg, and other open formats seem good, but they may hit a complexity wall. There's already some inconsistency between platforms, eg with delete vectors.
Incremental view maintenance interests me as well, and I would like to see it more available on different platforms. It's ironic that people use dbt etc. to test every little edit of their manually coded delta pipelines, but don't look at IVM.
This is a fairly simple idea of indexing characters for each column/offset and compressing the bitmaps. Simple is good, as the overhead of more sophisticated ideas (eg suffix sorting) is often prohibitive.
One suggestion is to index the end-of-string as a character as well; then you don't need negative offsets. But that turns the suffix search into a wildcard type of thing where you have to try all offsets, which is what the '%pat%' searches do already, so maybe it's OK.
It's been around for quite while, but DB people hate to explain where they got an idea. For all I know Vertica got it from somewhere else; I think postgres got jsonb around the same time.
My last dive into matrix computations was years ago, but the need was the same back then. We could sparsify matrices pretty easily, but the infrastructure was lacking. Some things never change.
I wrote part of that -- it needed a tight summary that covers all the main components, so I packed as much as I could into a sentence or two. I believe the second paragraph is mostly from me, but it was some time ago, back when Hanok revival was fairly new, and a lot of books were coming out on the subject.
A few years after that one of the Korean dailies wrote an English article and copied my wording. :(
This is fun, but I'm confused by the architecture. Duckdb is based on one-off queries that can scale momentarily and then disappear, but this seems to run on k8s and maintain a persistent distributed worker pool.
This pool lacks many of the features of a distributed cluster such as recovery, quorum, and storage state management, and queries run through a single server. What happens when a node goes down? Does it give up, replan, or just hang? How does it divide up resources between multiple requests? Can it distribute joins and other intermediate operators?
I have a soft spot in my heart for duckdb, but its uniqueness is in avoiding the large-scale clustering that other engines already do reasonably well.
I'm just learning about agentic search so I'm a bit adrift.
One of my side projects is a full text index for pattern search, and I'm trying to understand how it might fit with that. You mention tool call overhead, but is that a significant part of the latency in the multi-turn scenario, or is it the coding agent being forced into a serial processing pattern?
This article seems somewhat fanciful. Medieval Koreans ate two meals a day, and famines were common. Only a small area of the country is suitable for rice cultivation. The photos seem to show only upper-class clothing and furniture.
I've been to that makgeolli place in Jeonju, and it sells drinks and food as a set; there is no free food.
Nam (Man Tea) and other brands have been in convenience stores for years; most have a full shelf of it. As I heard it the male part of the name is due to its use in manly herbal mixtures, but it's not considered a male-specific substance on its own.
I built the ads audience system. Most of the effects of that are already known here; Ads became a big trophy within the org; everybody had to have ads and post-sale revenue, even the fridge people.
Sometime around when the CEO got out of prison a bunch of weirdness occurred. Good managers left, bad managers got hired, and everything became top-down. The group head "retired" but last year un-retired in a different position; I didn't know you could do that.
Engineering-wise it went from technical free rein to "only use this suspiciously chummy cloud vendor" in a few months. I never got to the bottom of that deal, but costs exploded, and revenue flattened.