This is really a question of economics. The biggest organizations with the most ability to hire engineers have need for technologies that can solve their existing problems in incremental ways, and thus we end up with horrible technologies like Hadoop and Iceberg. They end up hiring talented engineers to work on niche problems, and a lot of the technical discourse ends up revolving around technologies that don't apply to the majority of organizations, but still cause FOMO amongst them. I, for one, am extremely happy to see technologies like DuckDB come along to serve the long tail.
Not yet, but I believe the DataCouncil staff recorded it and will post it to their YouTube channel sometime in the next few weeks: https://www.youtube.com/@DataCouncil/videos
I just watched the author of this feature and blog post give a talk at the DataCouncil conference in Oakland, and it is obvious what a huge amount of craft, ingenuity, and care went into building it. Congratulations to Hamilton and the MotherDuck team for an awesome launch!
I agree 100% that this needs to be more of a thing. For data engineers building data pipelines, queries are like functions, and table schemas are like types. There needs to be a way to write a query that runs on an abstract interface, rather than an actual table. To do this, most folks rely on string templating in Python or Jinja, which makes the development process really cumbersome. As a result, most teams end up in scenarios where data pipelines are always a big mess of spaghetti SQL, or they are stuck maintaining complex frameworks that abstract away common logic, but are inscrutable to the average user.
The argument is more than that -- namely that in addition to having sophisticated AI, Google also controls the OS (Android) and hardware (Pixel). Being able to integrate best-in-class AI at every level of the stack is a tremendous advantage. OpenAI can't do this because they don't control the OS, and will always need to go through an app. Apple can play since they control OS and hardware, but at the moment they appear pretty far behind in the AI aspect.
He does a wonderful job of taking very dense mathematical notation and explaining it in ways that anyone can understand. He derives the basic concepts of the lambda calculus from the ground up using Python. Super fun to follow along with.
> I remember Tim Cook said they are not in the Ads Business. Unfortunately I am too busy, Apple's PR has worked hard to delete those tracks of what Tim Cook said, or Google's search engine is longer showing what I am looking for.
The first sentence of the actual article links to when Tim Cook said that.