loveparade

vor 9 Tagendiscuss

Fair point, I was more trying to make a statement about the amount of training data available, not the "difficulty" of the task. I just used Tailwind as an example because it is so ubiquitous with so much training data for LLMs to learn from, while any niche application doesn't have that.

loveparade

vor 9 Tagendiscuss

Used by Anthropic to sell tokens

loveparade

vor 9 Tagendiscuss

These studies are meaningless because speedup is heavily dependent on the kind of work you're doing. No doubt that you can do mechanical refactors 100x faster with AI, and also no doubt that using AI will be slower for tasks where it's less about writing code and more about context/world knowledge or building understanding. Averaging across these tasks doesn't make sense because everyone's work consists of a different distribution of tasks.

A frontend dev doing tailwind integration for his day job is gonna see very different speedups than someone working in a niche scientific codebase. Taking the average makes about as much sense as taking the average of the speedup from calculators for a mathematician, a farmer, and an elementary school student.

loveparade

vor 2 Monatendiscuss

Senior people get away with it because they understand the impact of their changes since they are intimately familiar with all the systems. So they know that their crappy shortcut doesn't break anything.

A junior employee doesn't have that knowledge, so they are expected the follow rules to ensure that they don't inadvertently break something.

loveparade

vor 2 Monatendiscuss

ai generated slop

loveparade

vor 2 Monatendiscuss

> If you used the same architecture as GPT2 today you're in for a bad time training a new frontier model. It's only because we have dozens of breakthroughs

What exactly are these dozens of breakthroughs? Most frontier models architectures today still look very much like GPT2 at their core. There were various of improvements like instructgpt, finetuning techniques, efficiency improvements with kv caches, faster attention, lora, better tokenizers, etc. Most of these are for making things run faster. The biggest differentiator has probably been data curation and post-training data and the ability to fit more into the model. But I think we had few breakthroughs that would fall into the category of different technologies.

loveparade

vor 2 Monatendiscuss

The more AI-generated AI bad stories we get the more likely LLMs will produce more!

loveparade

vor 2 Monatendiscuss

I haven't had a drink in 6 months or so. Not because I wanted to stop, I just have not had the desire to drink recently.

Now I would love to tell you about all the amazing magical healthy benefits that have come with that, but unfortunately there are none. I feel no difference at all.

loveparade

vor 2 Monatendiscuss

Doesn't matter who reads it. The point is that you will probably never learn to do "high level system design" well if you do not have enough experience writing and refactoring code yourself. It's like you wanting to become the chef of a kitchen and giving instructions without having ever prepped food.

There is indeed something useful about trying to write elegant code. Not because others read it. But because that's how you learn about the engineering tradeoffs and abstraction that exist everywhere.

loveparade

vor 3 Monatendiscuss

I give it one to two more years before open source models have fully caught up. Products are commodities and models are commodities too. GPUs cores are still hard to get for inference at scale right now. They need a platform with lock in but unsure what that would look like and why it wouldn't be based on open source models.

loveparade

vor 3 Monatendiscuss

Good lucking getting GPUs.

loveparade

vor 3 Monatendiscuss

Not really, when OpenAI was formed in 2015 there were no LLMs, at least none that worked well. It was a regular AI research lab mostly doing Reinforcement Learning on game environments like Atari similar to DeepMind. Once they struck gold with LLMs (2019 or so?) and saw there is money to made everything changed, as expected when a bunch of SV types get involved.

loveparade

vor 3 Monatendiscuss

I watched the video but I don't quite get it. I feel like I'm missing something? A nicer git workflow is not what I need because I can ask an LLM to fix my git state and branches. This feels a bit backwards. LLMs are already great at working with raw git as their primitive.

I'm curious what their long term vision they pitched investors is.

loveparade

vor 3 Monatendiscuss

Yeah, it looks like a model issue to me. If the harness had a (semi-)deterministic bug and the model was robust to such mix-ups we'd see this behavior much more frequently. It looks like the model just starts getting confused depending on what's in the context, speakers are just tokens after all and handled in the same probabilistic way as all other tokens.

loveparade

vor 3 Monatendiscuss

For me the reason to add dependencies to my projects is exactly because they are maintained upstream and I don't need to worry about maintaining them myself. If I need to fork and maintain it myself I'd rather write my own version of it that perfectly fits my use case, or use another dependency that is maintained.

loveparade

vor 3 Monatendiscuss

Understandable, I'd probably do the same in his position. Still sucks, we've seen this pattern a thousands times before and what happens next is pretty obvious.

I was prototyping something with pi under the hood for a personal project, going to switch off it now.

loveparade

vor 3 Monatendiscuss

Are you sure they are not just refusing to solve your UI bug due to safety concerns? They may be worried you'll take over the world once your UX becomes too good.

loveparade

vor 3 Monatendiscuss

It really seems it's mostly AI comments on this. Maybe this topic is attractive to all the bots.

loveparade

vor 3 Monatendiscuss

You could manage your subscriptions in an RSS reader, that's what I used to do. Each channel has multiple RSS feeds associated with it for different types of videos (live, vod, etc).

loveparade

vor 3 Monatendiscuss

The whole Youtube experience has gotten so bad over the years. I love the youtube content, but I wish I didn't have to deal with the UI/UX and recommendations that the YT app forces on me.

Annoying Shorts. I'm trying to keep my watch history clean to "steer" recommendations, but YT keeps adding things to it that I didn't actually watch just because I happened to hover my mouse over a video, etc.