lsy · HackerTrans

w zeszłym miesiącudiscuss

Yeah, claiming “product-market fit” on coding assistants for this multi-trillion dollar capital expenditure seems premature. Anthropic will post one and only one quarter of “operating profit” (aka losses after taxes and debt obligations) on the back of free-for-all spending by enterprise and engineer tokenmaxxing, neither of which will last. The investment was commensurate to a world-eating AGI, and if all that comes out of it is coding agents and slightly better enterprise software, I don’t think that makes up for the money spent.

lsy

2 miesiące temudiscuss

There's a very real possibility that AI proponents completely lose the next generation of adults. The output is not enjoyable to consume, the people who rely on it are not cool, and the effects of using it are unpleasant and hard to defend on aesthetic, intellectual, or moral grounds.

There are real use cases for this technology! But the idea that the generation of superficially plausible text is "the next Industrial Revolution" comes out of the same mindset that has turned a neat technology into a banal hellscape for consumers and employees. We desperately need some leadership in companies or institutions that can place this technology in its proper context, and leverage it without getting manic about it.

lsy

3 miesiące temudiscuss

AI coding isn’t an abstraction, though. You can’t treat a prompt like source code because it will give you a different output every time you use it. An abstraction lets you offload cognitive capacity while retaining knowledge of “what you are doing”. With AI coding either you need to carefully review outputs and you aren’t saving any cognitive capacity, or you aren’t looking at the outputs and don’t know what you’re doing, in a very literal sense.

lsy

5 miesięcy temudiscuss

At what point do LLMs enable bad engineering practices, if instead of working to abstract or encapsulate toilsome programming tasks we point an expensive slot machine at them and generate a bunch of verbose code and carry on? I'm not sure where the tradeoff leads if there's no longer a pain signal for things that need to be re-thought or re-architected. And when anyone does create a new framework or abstraction, it doesn't have enough prior art for an LLM to adeptly generate, and fails to gain traction.

lsy

5 miesięcy temudiscuss

It's easy to get this way with enough scrolling, try to focus on the things around you in real life. If you aren't reading LinkedIn or HN, how much do you actually hear about AI in day-to-day life? If someone at work directly asks you to do something using AI, you might make some effort to do it. But otherwise let the news and hype cycle play out. You don't need to anticipate or keep abreast of where people think things will be in ten years... they are almost certainly wrong. Think of LinkedIn and HN as entertainment at best. Work on personal coding projects without AI, build relationships with non-tech people, go outside.

lsy

5 miesięcy temudiscuss

It’s notable that just the English “implementation” of FizzBuzz here is longer and more ambiguous than the naive Python implementation, never mind the boilerplate (which itself is also longer than the Python).

The explosion of frameworks and YAML tools the author describes can be attributed to the fact that English is an extremely poor language for program specification, and requires all kinds of guardrails and annotation to accomplish the same specificity as a typical computer program.

lsy

5 miesięcy temudiscuss

LLM coding isn't a new level of abstraction. Abstractions are (semi-)reliable ways to manage complexity by creating building blocks that represent complex behavior, that are useful for reasoning about outcomes.

Because model output can vary widely from invocation to invocation, let alone model to model, prompts aren't reliable abstractions. You can't send someone all of the prompts for a vibecoded program and know they will get a binary with generally the same behavior. An effective programmer in the LLM age won't be saving mental energy by reasoning about the prompts, they will be fiddling with the prompts, crossing their fingers that it produces workable code, then going back to reasoning about the code to ensure it meets their specification.

What I think the discipline is going to find after the dust settles is that traditional computer code is the "easiest" way to reason about computer behavior. It requires some learning curve, yes, but it remains the highest level of real "abstraction", with LLMs being more of a slot machine for saving the typing or some boilerplate.

lsy

5 miesięcy temudiscuss

I think the analogy to high level programming languages misunderstands the value of abstraction and notation. You can’t reason about the behavior of an English prompt because English is underspecified. The value of code is that it has a fairly strong semantic correlation to machine operations, and reasoning about high level code is equivalent to reasoning about machine code. That’s why even with all this advancement we continue to check in code to our repositories and leave the sloppy English in our chat history.

lsy

8 miesięcy temudiscuss

It's disheartening that a potentially worthwhile discussion — should we invest engineering resources in LLMs as a normal technology rather than as a millenarian fantasy? — has been hijacked by a (at this writing) 177-comment discussion on a small component of the author's argument. The author's argument is an important one that hardly hinges at all on water usage specifically, given the vast human and financial capital invested in LLM buildout so far.

lsy

9 miesięcy temudiscuss

Going to a popular restaurant that accepts app delivery orders (or a grocery store in a neighborhood where people prefer to pay for delivery) is an objectively bad experience. The kitchen or checkout line is backed up with delivery orders, there are a bunch of delivery drivers double-parked or loitering near the front, and due not to any moral failing but rather what must be a crushing grind, the drivers are for the most part rushed and inconsiderate of the staff or other customers.

The class of people who order delivery regularly are generally trading the short-term reward of convenient food for way more money than makes sense, too little of that money benefits the class of people who do the delivering, and as the article points out, it is essentially harming the business it's being ordered from.

I would love to see more restaurants and stores declining to support this kind of system. While there may be some marginal profit now, in the long run the race to the bottom is going to mean fewer sustainable businesses.

lsy

9 miesięcy temudiscuss

I feel like this needs an editor to have a chance of reaching almost anyone… there are ~100 section/chapter headings that seem to have been generated through some kind of psychedelic free association, and each section itself feels like an artistic effort to mystify the reader with references, jargon, and complex diagrams that are only loosely related to the text. And all wrapped here in a scroll-hijack that makes it even harder to read.

The effect is that it's unclear at first glance what the argument even might be, or which sections might be interesting to a reader who is not planning to read it front-to-back. And since it's apparently six hundred pages in printed form, I don't know that many will read it front-to-back either.

lsy

9 miesięcy temudiscuss

It's interesting to call this a pre-mortem as it seems mainly organized around thinking positively past the imperfections of the technology. It's like a pre-mortem for the housing crisis that focuses on the benefits of subprime mortgage lending.

What I'd expect to see is an analysis of how to address or prevent the same situation as previous bubbles: that society has allocated resources to a specific investment that are far in excess of what that investment can fundamentally be expected to return. How can we avoid thinking sloppily about this technology, or getting taken in by hucksters' just-so stories of its future impact? How can we successfully identify use-cases where revenues exceed investment? When the next exciting tech comes around, how can we harness it well as a society without succumbing to irrational exuberance?

lsy

9 miesięcy temudiscuss

I think this leaves out what is probably the most likely future for this technology, having a similar destiny to most technologies as a tool. Both of these visions assume (I think incorrectly) a trend towards ubiquity, where either every interaction you as a person have is mediated by computers, or where within a certain "room" every interaction anyone has is mediated by computers.

But it seems more likely that like other technologies developed by humanity, we will see that computers are not efficient for, or extensible to, every task, and people will naturally tend to reach for computers where they are helpful and be disinclined to do so when they aren't helpful. Some computers will be in rooms, some will get carried around or worn, some will be integrated into infrastructure.

Similar to the automobile, steam powered motors, and electricity, we may predict a future where the technology totally pervades our lives, but in reality we eventually develop a sort of infrastructure that delimits the tool's use to a certain extent, whether it is narrow or wide. If that's the case then the work for the field is less about shoving the tech into every interaction, and more about developing better abstractions to allow people to use compute in an empowering rather than a disempowering way.

lsy

9 miesięcy temudiscuss

No doubt it's a profit margin game, but I wish the big e-reader companies (Kindle, Kobo) would take a foray into this form factor. The friction of navigating through an Android interface into an app is just enough to negate the convenience benefit of a pocketable device. But the mainstream e-readers are unfortunately just big enough to require a jacket or a bag to carry them in.

lsy

9 miesięcy temudiscuss

I'm sure it's nearly an academic distinction, but:

> Basically, for any given region, we find its highest point and assume that there is a perfectly placed sibling peak of the same height that is mutually visible.

Shouldn't you always add 335km to the horizon distance to account for the possibility of Everest (i.e. a taller sibling peak) being on the other side of the horizon?

lsy

9 miesięcy temudiscuss

Impressive that this was done in 3 days at all, but to anyone who is familiar at all with System 7's appearance, the screenshot is almost comically "off" and gives away that this is not a straight port so much as some kind of clean-room reimplementation. The attached paper is more reserved, calling this a "bootable prototype".

lsy

10 miesięcy temudiscuss

Fixing "theoretical" nondeterminism for a totally closed individual input-output pair doesn't solve the two "practical" nondeterminism problems, where the exact same input gives different results given different preceding context, and where a slightly transformed input doesn't give a correctly transformed result.

Until those are addressed, closed-system nondeterminism doesn't really help except in cases where a lookup table would do just as well. You can't use "correct" unit tests or evaluation sets to prove anything about inputs you haven't tested.

lsy

10 miesięcy temudiscuss

A world model itself, in its particulars, isn't as important as the tacit understanding that the "world model" is necessarily incomplete and subordinate to the world itself, that there are sensory inputs from the world that would indicate you should adjust your world model, and the capacity and commitment to adjust that model in a way that maintains a level of coherence. With those things you don't need a complex model, you could start with a very simple but flexible model that would be adjusted over time by the system.

But I don't think we have a hint of a proposal for how to incorporate even the first part of that into our current systems.

lsy

2 lata temudiscuss

This has always been the end-game for the pseudoscience of "prompt engineering", which is basically that some other technique (in this case, organizational policy enforcement) must be used to ensure that only approved questions are being asked in the approved way. And that only approved answers are returned, which of course is diametrically opposed to the perceived use case of generative LLMs as a general-purpose question answering tool.

Important to remember too, that this only catches those who are transparent about their motivations, and that there is no doubt that motivated actors will come up with some innocuous third-order implication that induces the machine to relay the forbidden information.

comments