Perhaps my "reject the premise" comment was a bit too provocative. I didn't intend to start an argument. I wanted to share a story of a person who was put in the position you described (working as a manager on a large project, required to have an overview of the project but not necessarily to actually write or review much code), failed miserably, changed the rules of engagement (partly by reviewing code), and subsequently succeeded. So my _personal_ answer to your question of how to build a mental model in that scenario would be to do something in the same vein as that story. Not necessarily _exactly_ what that guy did, but I think the principles still apply today. Nothing really changes.
I am not able to find it now, but there was an amazing story recently from the 60s or 70s where an engineer was in exactly this position. His team was building a new, complex, ambitious operating system, but it was late and over budget and didn’t work. It nearly wrecked the company. He talks about hitting rock bottom and asking himself what went wrong, and one of the fathers of computing (can’t remember which) shouts from the hallway in passing, “that’s easy, you didn’t understand what your people were doing.” So the guy turned it around by implementing a new rule: he had to understand every line of code his team wrote. They started over with the company’s existing OS in use by customers and implementing a few of the most requested features. Much less ambitious, but it actually shipped. Gradually they achieved all their goals by upgrading the existing system.
The “I must understand every line” constraint didn’t sound like a power trip that succeeded because the guy was such a brilliant code reviewer. I think it was a blunt instrument that enforced simplicity.
I guess what I’m saying is, I reject the premise of having technical oversight without writing or reading much code.
Folks have been saying “things are different now, the agents are now compounding success instead of error” for at least a year now, but I just don’t see it. I was lucky enough to receive a weeklong $50k per head AI training from the people saying these things, and one of their few helpful concrete recommendations was to constantly clear context all the time, to avoid things going off the rails.
However, I think finding security vulnerabilities is one use case where it doesn’t matter. Tokenmaxxing is absolutely effective for that. We as an industry are in the middle of adopting very expensive, complex continuous fuzzers.
> prompting for test-driven development is not the same as enforcing code coverage thresholds in your build tool
Are they actually different? I would guess they have roughly the same efficacy. 100% code coverage means nothing, and this is especially true with LLMs.
Oh no, I’ve definitely experienced that, it’s terrible. But that situation makes me wish for more agency (for example, talking to customers directly), whereas it seems to make other engineers wish for less agency (please hand me a complete spec and I will mindlessly translate it to code). That’s what I don’t understand.
I’ve never understood engineers who complain about vague specs… if the spec was complete, it would be code and the job would be done already! Getting a 20 page spec delivered from upon high and mechanically translating it to code without any chance to send feedback up the chain sounds like… a compiler.
The saddest part about Kubernetes is… after you set it all up, you still need a hacky deploy.sh to sed in the image tag to deploy! And pretty soon you’re back to “my dear friend you have built a Helm”. And so the configuration clock continues ticking…
Jujutsu has a concept of mutable vs immutable commits to solve this. Usually everything in a remote branch is immutable. To work on a branch, I track it and that makes it mutable.
I’ve been driving Bluefin DX for a year or two. On the plus side, it works absolutely flawlessly. This is the longest I’ve ever run a Linux distro without a Nvidia driver update causing the whole thing to explode. It truly is the year of Linux on the desktop.
But I can’t say I recommend it for dev work. It wants you to do everything inside devcontainers, which I like in theory but in practice come with so many annoyances. It wants you to install Flatpaks but Flathub is pretty sparse. I ended up downloading raw Linux binaries into my home directory (which actually works surprisingly well. Maybe this is the future, hah)
I think next time I’ll just go with vanilla Fedora.
No encapsulation… huge functions with tons of local variables shared between closures… essentially global state in practice. I think ant the time, objects with member variables felt “heavy” and local variables felt “light”. But the fact that they were so lightweight just gave me more opportunities to squirrel away state into random places with no structure around it. It really wasn’t all that horrific, and it helped me ship something quickly, but it wasn’t maintainable. These days I think the “heavy boilerplate” of grouping stuff into structs and objects forces me to slow down and think a bit harder about whether I really want to enshrine a new piece of state into the app’s data model. Most of the time I don’t.
I also think there’s an interesting effect when cool functional language features like currying and closures are adopted by imperative languages. They make it way too easy to create state in a way that makes you FEEL like you’re writing beautiful pure functions. Of course, in a functional language everything IS pure and this is just how things work. But in an imperative language you can trick yourself into thinking you’ve gotten away with something. At one point I stored practically all state in local variables captured by closures. It was a dark time.
Right, there’s no silver bullet. I think all I can do is increase the feedback bandwidth between my brain and the real world. Regular old stuff like linters, static typing, borrow checkers, e2e tests… all the way to “talking to customers more”
I think the worst case is actually that the LLM faithfully implements your spec, but your spec was flawed. To the extent that you outsource the mechanical details to a machine trained to do exactly what you tell it, you destroy or at least hamper the feedback loop between fuzzy human thoughts and cold hard facts.