Slack is such a simple product, and is so strategic as an interface for AI, Anthropic has to be considering building their own Slack. Hopefully this tag approach is an MVP and it proves the potential of workplace messaging as an interface for AI, but also the limitations of relying on the extension points they salesforce chooses to provide, and it turns into a full fledged slack competitor from anthropic soon.
Claude-in-Slack is a big enough feature to overcome the slack-connect network effect. Openness is absolutely key! I wrote this post because I hoped that if Anthropic is already planning to do this I might be able to influence them to make open-data part of the plan. But openness by itself isn't a big enough feature to get users.
Also true! The most important thing is that the NewSlacks commit to interoperability. I think Anthropic has a special opportunity to lead the way here, because they have a track record of standing by their principles to an extraordinary degree.
You can only access public channel data, you can't even access that at scale, and Claude needs to be more natively integrated in ways that Slack will never allow.
I talked to the timescale CTO at pg conf a few years ago and asked him what timescale does differently than a standard columnar database that makes it better suited for time oriented data. He said a bunch of things and I said “but columnar databases do those things.” Then he got mad at me.
I guess it’s just another columnar dbms after all?
They make a really good criticism of Iceberg: if we have a database anyway, why are we bothering to store metadata in files?
I don’t think DuckLake itself will succeed in getting adopted beyond DuckDB, but I would not be surprised if over time the catalog just absorbs the metadata, and the original Iceberg format fades into history as a transitional form.
This is exactly right. We even went so far as to build a proof of concept internally, and the technical challenges are just very different. The simplest way to explain it is that Fivetran connects a skinny pipe (APIs) to a fat pipe (databases) while Census connects a fat pipe to a skinny pipe.
I am generally a huge vertical sharding skeptic but there are special cases where it is beneficial. If you have a simple query pattern on one table that represents a big fraction of your entire workload you can put it into its own instance and it becomes much easier to monitor. It’s easy to see why vertical sharding is sometimes the right answer by inverting the decision: should we put two unrelated large applications on the same instance? Obviously not, there is no benefit and ops becomes more difficult.
Like so many things from Google engineering this will be toxic to your startup. SREs read stuff like this, they get main character syndrome and start redoing the technical designs of all the other teams, and not in a good way.
This phenomenon can occur in all “overlay” functions, for example the legal department will try to run the entire company if you don’t have a good leader who keeps the team in their lane.
In general, I absolutely agree with you. It’s basically an instance of “the customer is always right”: if a smart customer can’t get our product working, there is a problem with the product. But this post made a much bolder (and wrong) claim: “the product has a number of major design flaws that mean that it literally cannot work”.
I have some insight into this because this claim is about my company Fivetran:
“…relies on the data source being able to seek backwards on its changelog. But Postgres throws changelogs away once they're consumed, so the Postgres data source can't support this operation”
Dan’s understanding is incorrect, Postgres logical replication allows each consumer to maintain a bookmark in the WAL, and it will retain the WAL until you acknowledge receipt of a portion and advance the bookmark. Evidently, he tried our product briefly, had an issue or thought he had an issue, investigated the issue briefly and came to the conclusion that he understood the technology better than people who have spent years working on it.
Don’t get me wrong, it is absolutely possible for the experts to be wrong and one smart guy to be right. But at least part of what’s going on in this post is an arrogant guy who thinks he knows better than everyone, coming to snap conclusions that other people’s work is broken.
ksql does not solve any of the hard consistency or contention problems you will face if you attempt to use Kafka as a datastore. Consider the simplest possible example: you write an "update event" to a topic and then read a ksql view of that topic. The view may or may not yet reflect the update. This is called read-after-write consistency, and you will need to create it in your application code.
This notion of “stream-table duality” might be the most misleading, damaging idea floating around in software engineering today. Yes, you can turn a stream of events into a table of the present state. However, during that process you will eventually confront every single hard problem that relational database management systems have faced for decades. You will more or less have to write a full-fledged DBMS in your application code. And you will probably not do a great job, and will end up with dirty reads, phantoms, and all the other symptoms of a buggy database.
Kafka is a message broker. It’s not a database and it’s not close to being a database. This idea of stream-table duality is not nearly as profound or important as it seems at first.