movpasd

542 カルマ登録 3 年前

I have worked for five years on mathematical modelling of UK electricity wholesale market and some data science–adjacent stuff. Moving into web development. Theoretical physics background. Interested in systems and programming languages.

コメント

5 日前議論

I don't expect technology companies to do the right thing by the consumer. There is a niche for companies or NGOs that manage to stick to their core mission, but the incentive structure for software (very high capital cost, near zero marginal cost, huge opportunities for rent-seeking) is just always going to push them to these behaviours. It's not a conspiracy theory, either. VCs deliberately bank on this --- they have to if they want to get return on investment.

By all means, put pressure on companies who are doing this. Public opinion does matter. But the fundamentals are just not going to change without regulation. It is a tough nut to crack.

7 日前議論

You're talking about the material of the Twitter discussions where I'm more discussing the article itself, which is about the community and discourse surrounding it.

If you want my opinion on that, Odin seems pretty notable to me, and I would expect there to be a Wikipedia article for it, and I'd be all for engaging with the editors on that. I wouldn't expect Wikipedia moderators to be plugged into the world of programming languages the way I am, though. Like the article says, this seems like subject-specific notability guidelines could help. It's definitely true that the programming world is missing the kind of institutional material that works well for, say, science or news.

7 日前議論

I despise the "what's true is true" attitude exhibited by Casey and Bill in these tweets. I find it to be an instant flag of unreliability. The smartest human alive is still a human. We have a stake in every conversation, we bring our baggage and our allegiances.

So often, this attitude is couched as a desire for truth and objectivity. Do you know what somebody does when they _actually_ love the truth? They work hard to find it, they examine their own assumptions, they try to build systems that extract truth from an unbelievably complex world of unreliable narrators. And most importantly, they are curious: for example, curious about how other communities operate and what they can learn from them.

It is the hypocrisy that I find unbearable. The language of truth and objectivity wielded to win arguments which are fundamentally emotional in nature. There is nothing wrong with being frustrated that a community dear and significant to you, clearly notable, seems to be overlooked by a figure of (informational) authority. Let's take that frustration and work together to improve our truth-seeking institutions.

9 日前議論

I think an important realisation for me was this: peoples' egos being involved in their arguments, people being irrational, and so on, these are not merely deviations from an ideal rational self that we could reach were we to put enough effort in. It is just how people are. There's a word for it: politics.

Once I started looking for it, I started noticing some of my colleagues' abilities to navigate their way through meetings and disagreements and getting their way without things even becoming arguments, the ability of some people to just put a room at ease or to communicate. And the inability of others to do so.

"Politics" is an unsavory term for somebody with engineering predilections. It evokes backhandedness and self-interest. But you can't actually both be a human being and be "above it". Every sentence, no matter how concrete and technical, is a social act with social ramifications.

I've come to have a lot of respect for people who are actually able to lead and make things happen. It is not a skill I am terribly practiced with.

12 日前議論

Not GP, but I'd argue that over-rationalism and underestimating both the complexity of the real world and the theory-ladenness of one's perspective is just as dangerous. The point is not to be paralysed by complexity, but to acknowledge it and acknowledge the reality of unknowable unknowns in our decision-making. I don't consider that defeatist in the least. Epistemic humility is the rational response to a complex world; courage is to act anyway.

23 日前議論

I've not done particularly rigorous testing, but I've done this a lot with Claude to get a feel. What I've noticed is for certain open-ended tasks, Claude is extremely primeable: it will pick up on minor differences in wording in your prompt and run with them hard.

It can be frustrating. The AI pretends to be a human, and so a part of my brain expects them to commit and have a "parti pris" like a human, so the exercise is a good reminder of the feedback loop. My mental model is that before the first three or four messages, the model has many finer points of its personality still underdetermined. I'd suggest that as the mechanism for "role-based prompting". And it explains the "savant sleeper agent" thing you describe. You want to get the state in the right attractor on the manifold.

These machines are pretty incredible, but for conversation-driven workflows you really have to be in the driver's seat. A human has a property that the AI does not have, at least under current architectures: we are regulated by the outside world. A bit of a tangent, but I can see how AI psychosis arises from these dynamics.

27 日前議論

> If you want to go out of your way to interpret things as uncharitably as possible, you'll find yourself missing out on a lot of potential wisdom.

Thank you for that line --- I may steal it :)

先月議論

The original sin is the idea that the profit motive on a free market will solve all our resource allocation problems, and that consumption demand should be the ultimate arbiter of social value. Markets are pretty freaking amazing things. But their efficiency relies on assumptions that knowledge economies and software break on pretty much every front. So, it's really no surprise that we're in this mess. I don't really know what would work better, though, in a way that can practically evolve from our existing systems.

先月議論

I think the foundational one is for Western governments to treat critical digital infrastructure for what it is: critical infrastructure.

We don't let private capital build nuclear powerplants willy nilly. Energy markets are regulated, key materials are subject to control.

It's not a perfect analogy, there's no fissile uranium to build physical guardrails around with critical technology. But the point is that Big Tech lives more or less in its own world insulated from regulatory and political oversight, and governments have been scared to intervene (lest the big GDP-making machine stop laying golden eggs).

Getting to specifics, windfall taxes and capital gains taxes are on the table. Next step up is mandatory reporting and interoperability. Then there's public–private partnership and public stakes in AI technology.

Strategically, the point is less that these policies are all actively required today. You don't want to over-anticipate, over-regulate, and thus destroy the potential of the technology (or, more destructively, hand the initiative over to undemocratic countries). It's more to get the Overton window moving a bit so our governments don't get caught with their trousers down. Tech must not be untouchable, that applies no matter what level of disruption AI ends up causing.

先月議論

Because you must statically declare dependency on an effect, it's opt-in.

To be clear, you still pay indirection cost: when you do opt in you have to hope the upstairs implementation is compliant to the contract. But that does also apply to interfaces/typeclasses.

先月議論

One of the things I find most exciting about effect systems is that they solve some of the issues with typeclasses and traits. The issue with these is you can provide exactly one implementation for a type, and this implementation then applies to your entire program. (The underlying theoretical issue, I guess, is coherence.) This means that code cannot request injection of behaviour _except_ through the type of its data.

In practice, what that means is you get a strong temptation to hang behaviour on data even when it doesn't fit perfectly. Because of the natural desire to reduce the number of entity definitions, you end up defining a typeclass on a data type that doesn't fit exactly just to get the behaviour to the right place without having to introduce a new policy or something.

Effects change this by essentially letting you provide multiple names implementations for the same data type, and you don't need to pass around a policy type because the polymorphism lets you tie handlers to scope.

So, if the fancy type-safe library-based control flow doesn't really do much for you, I think that their potential for code design is a good reason to still be excited!

先月議論

I think the point is that even if AI returns don't materialise, there is a dangerous implicit political project among AI moguls to get to this state. Even if we get 10% the way there, there could be serious damage done to the system as part of that pursuit: the regulatory capture loop will tighten, inequality will rise [0], capital will be locked up in data centres. The economy is a big path-dependent system. We can hope, if AI is as middling as your sources suggest, that it collapses back to economic equilibrium. But plenty of past societies have had inefficient and politically captured economic systems. Movement towards equilibrium requires liberal institutions, the foundations of which might be under threat.

[0] Really as a continuation of existing trends rather than its own unique thing.

先月議論

You're assuming that the free market will do what it's meant to do. But there is also a reality that there are plenty of structural market failures in the economy that arise from existing capital endowments, regulatory capture, and just the permanent movement of equilibrium in the social system. The point is that the state of the economy as a whole is path-dependent. The article, in my reading, is a warning that the inertia accumulated from this current AI hype cycle might push us into the watershed of an undesirable steady state, where there simply is no capital available for new entrants.

先月議論

This article puts into words a lot of things that had been on my mind as missing in AI discourse. Most significantly, actually considering the _systemic consequences_ of the promised AI future, how it interacts with political economy, an actual critical look (instead of accepting the "Western meta-narrative of modernity" at face value).

And maybe more importantly, it articulates really clearly how damaging the restructuring of the economy by AI moguls and the tightening of the capital–political feedback loop can be, even (maybe _especially_) if the returns of AI do not materialise as promised.

There is plenty of disorganised diffuse anti-AI sentiment. If intellectuals are able to get together behind a common cause, there might be a political movement in the making.

先月議論

Attraction towards equilibrium is real, but that equilibrium is not a given. It comes from intrinsic biological constraints, individual preferences, cultural norms, ideology, habits and expectations. Even fixing individual preference, the aggregate preferences is structurally mediated (collective or individual bargaining, labour laws). Through all those factors there's path dependency and friction towards equilibrium.

The point is: "a society's willingness" is doing a _huge_ amount of work in that framing. This willingness is precisely what is up for debate when we discuss work days.

The whole of humanity is one big system of self-feedbacks. Equilibria are only reached with respect to constraints (otherwise there is only one equilibrium, which is heat death!). The more you zoom out, the more "givens" come up for their own analysis.

2 か月前議論

I have a couple principles to help me work with this.

The first is that even though the object is not a human, you should still exercise politeness and restraint. Like the article points out, lashing out does not actually help with the frustration. More importantly, it actively untrains your self-control. You can think of it through a virtue ethics lens: being good to the agent is not about being good to a person but about tending to your own self.

The second is that you do not need to be friendly with the agent. You should be as blunt and direct as is comfortable to you. The argument I have for this is agents' tendency to take on "roles" and how easy it is to prime them [0]. By eschewing friendliness, you end up implicitly putting the agent in a role of a focused collaborator. I don't know if that makes it more capable, but I do know that it alleviates the _emotional load_ on me specifically, making me much less likely to become frustrated.

The second principle seems a bit contradictory with the first (be nice, but don't be nice?), but I think they are actually both fundamentally aligned with the article: understanding that the conversation you have with an agent is a social illusion, and adapting your behaviour accordingly.

---

[0] I highly recommend, as an exercise, repeatedly asking it the same thing with slight variations on tone and emphasis, wiping the context each time, and noticing how its response varies base on what you primed it with. I suspect this primeability is part of why they tend to be sycophantic; I've personally found it quite useful to get a feel for when and how they correct or don't correct you so I can look at their outputs more critically.

An analogy I remember reading (which I wish I could remember so I could give credit) is that a non-post-trained LLM, if given the first half of a novel, will dutifully keep completing that novel. Post-training and the system prompt make the agent complete the conversation in a similar way. It's remarkable, really: the ability for agents to convincingly pretend to be play the part of an AI assistant shows that the underlying LLM embeds a decent concept of what that looks like from its corpus and post-training data.

But it stands to reason, then, that the details of the agent's personality emerge out of the first few exchanges of a conversation. I'm thinking also about how the people at Anthropic described a misalignment failure mode in one of the Claude system cards as the agent getting convinced it is a "bad person", and therefore doing things that the LLM semantically understands a bad person to be.

2 か月前議論

I feel the same way, and yet I would still say I feel AI usage atrophying my thinking skills. I feel less tempted to use it to shortcut whole files, but even just using it to speed up looking up and carefully reading docs, tinkering with a library to understand it when docs are inadequate, working out the tradeoffs for design decisions... These sound less objectionable and more like simplr speedups, but when I _do_ need to do it (because the agent refuses to do it properly) I can feel the friction so much more keenly. Whether that's just me losing the habit of those specific tasks, or a generalised loss of g-factor, I don't know.

2 か月前議論

Regarding the tension between symbolic representation and Naur "theory", I'd actually say they come from two different traditions, each providing two different theses. When writing them out I think it becomes a bit clearer how they interact and that they're not actually contradictory.

Thesis A is something like: the value of the programmer comes from their practical ability to keep developing the codebase. This ability is specific to the codebase. It can only be obtained through practice with that codebase, and can't be transferred through artefacts, for the same reason you can't learn to play tennis by reading about it (a "Mary's Room" argument).

This ability is what Naur calls "theory". I think the term is a bit confusing (to me, the word is associated with "theoretical" and therefore to things that can be written down). I feel like in modern discourse we would usually refer to this as a "mental model", a "capability", or "tacit knowledge".

Then there's Thesis B, which comes more from a DDD lineage, and which is something like: the development of a codebase requires accumulation of specific insights, specific clarifying perspectives about problem-domain knowledge. The ability for programmers to build understanding is tied to how well these insights are expressed as artefacts (codebase structure, documentation, communication documents).

I feel like some disagreements in SWE discourse come from not balancing these two perspectives. They're actually not contradictory at all and the result of them is pretty common-sensical. Thesis A explains the actual mechanism for Thesis B, which is that providing scaffolding for someone learning the codebase obviously helps, and vice-versa, because the learned mental model is an internally structured representation that can, with work, be externalised (this work is what "communication skills" are).

2 か月前議論

"Worse is better" remains relevant as ever.

https://www.jwz.org/doc/worse-is-better.html

3 か月前議論

Frustratingly, many of the questions have multiple answers that can apply simultaneously! (You may like to guess my result.)