My side project is now codebase explainability. I basically don't buy the premise that we just have to give up on comprehension as code generation scales; I just think that text is too limited by itself. So going a step deeper than asking Claudex "teach me this project", but having it produce a navigable snapshot of what's going on.
Big bang prototypes have been pretty awful, even after feeding the LLMs huge documents / wishlists / descriptions of how it should work, etc. Part of the experiment was giving LLMs some leeway to make product decisions with a lot of north star guidance, but AFAICT they are really bad at this. I also tried basic bottom-up efforts, which have been better but obviously more tedious. Now I'm trying to find a more scalable bottom-up approach that is more LLM-accelerated
Because for its entire existence, the top HN comment on articles is typically a contrarian take or pointing out flaws. This goes double for a study, where people just hunt for some aspect of the methodology they dislike. If you don't address the flaws, then it looks like you never considered them, and the top comment will say that your entire methodology is suspect. It's super predictable to the point that you can harness this kind of reaction to get stuff on the frontpage if you really want to.
Nah, "Any other vector DB" starts to fall apart once you need stuff like scripted scoring like OP uses. Then it starts to be a question of, "do you need ANN for performance?" since SQLite only does brute-force vector scoring. And granted, brute-force is performant for far more vectors than most people give it credit for, but it definitely hits a wall well below 1 million if you want it to have webpage-type latency.
Maintaining Elasticsearch isn't free, but picking an underpowered db and having to port to the right one is also quite time consuming.
I needed to set up a VPS for my blog, so I gave Claude SSH access to a fresh Digital Ocean droplet and told it to set up a deploy with Ansible and harden the instance. It one-shot it in 15 minutes.
That is indeed how things like copy/paste through the menus already work; you install an extension to get the clipboard permission or the browser limits you to keyboard shortcuts
Applications like Google Docs would be impossible without each of the four things you listed being available. We had Google Docs that didn’t roll-your-own for each of the items in that list and it was called Writely and it was absolute dogshit (but super cool for its time) because those limitations were too extreme. And by extension it wouldn’t make sense to have Chromebooks as a category of hardware, because web software could never compete at a feature parity level with native software.
> The decision was made to go with Eclipse. Then it magically went into some sort of internal box/decision process, and came out IntelliJ instead. I've always thought this was because of a sufficiently highly placed Android person with a personal preference, but I could be 100% wrong.
I'm relying purely on memory, but all of the reasons were super public at the time. The internal Java editor tooling team decided that they weren't investing any more time into Intellij. And then a few months later, the Android team relaunched Android Studio using Intellij with full internal support. The existing Java tooling squad put up some amount of fuss saying that they had already made the decision, and the Android team gave a long and diplomatic response that could be compressed down into, "we don't report to you and we don't care what you think." This coincided with the entirety of Google becoming mobile native, so the mobile team were given free rein to call the shots.
> IMO IntelliJ never worked very well in google3
YMMV, we had some people in Docs/Drive who invested a lot of time into making Intellij usable for everyone, and following their system, it worked way better for us than Eclipse did. All of the "Let me sing you the song of my people ::UI freezes::" memes were very true in my experience.
Yeah, a lot of people came of age with a "we'll fix it when it's a problem" mindset. Previously their codebases would start to resist feature development, you'd fix the immediate bottlenecks, and then you could kick the can down the road a bit until you hit the next point of resistance. You kinda refactor as you do features. The frontier models have pushed the "it's a problem" moment further back. They can kinda work with whatever pile of code you give them... to a point. So it manifests as the LLM introducing extra regressions, or dropping more requirements than it used to, but it's not really manifesting as the job being harder for you. It's just not as smooth as it was from an empty repository. Then you hit the point where it just breaks too much and you need to fix it. And the whole codebase is just fractal layers of decisions that you didn't make. That's hard to untangle. And you're not editing the code yourself, so you don't have that visceral "adding this specific thing in this specific way has a lot of tension" reaction that allows you to have those refactoring breakthroughs.
The proof is in the pudding. At this point, there have been plenty of models that overperformed on benchmarks and underperformed on real work. So my stance is that I'm curious, I'm excited to see where it goes, and I don't believe it until I can try it.
Yeah, this is exactly what I thought when I read this post. It seemed like the author either hasn't worked in big tech, or hasn't worked in the industry very long. It's extremely likely that the engineer who designed this was standing on his desk shouting "it's going to cost THIS MUCH MONEY. I want to make sure that EVERYONE IS OK WITH THIS." and was met with shrugs.
Here's how a big tech reporting chain sees this situation when everything is smooth sailing: "We're growing 3x year-over-year? After 2 years, the cost will be an order of magnitude higher no matter what solution we pick. The constant factor doesn't matter that much. But we have such an incredible roadmap that we will book more than an order of magnitude of revenue, backed by this new ledger project. The cost will always be a nonissue because of growth."
And then 2 years go by, and this incredible product growth adds a bunch of ledger entries that weren't there 2 years ago, someone nudges your reporting chain with the question, "this is pretty expensive.. what gives?" and then someone with a good combination of social and technical skills points out that a migration to your existing storage solution would be a cost effective way to continue growing.
At every step of the way, everyone is generally happy with what's going on.
And even still, it’s hard to ignore him because he’ll still have some insight like “token efficiency is going to be the big thing we care about in the future,” which was a small section of one of his Gastown blogs. And after you’re done asking yourself if that means Gastown is performance art, or he just said that as a hedge against his “vomit tokens” approach so that he could say “look I knew it all along” when this is all proven to be misguided, you’ll start to mull on just the concept of “token efficiency” and realize “someone who can do work in 1/10th the tokens will be king.” I mean hell, Gastown preceded real support for multiagent orchestration in Claude and potentially nudged it along that path.
I have some blog posts coming out soon. I’m also trying an experiment where I make YouTube videos[0] on each of them. My first video was a huge lift, since it was my first time doing everything.
Random observations from my first one:
- presenting my idea visually helped crystallize my thinking in a way that writing doesn’t. And writing was already very good at crystallizing my thinking.
- even making a bad video was a lot of work
- making a video presentable is a deep subject. Subtle changes were throwing off my setup. Now I understand why so many influencers are fitness and lifestyle; the demand side is obvious, but when you’re already camera-ready you have a huge advantage on the supply side
- described something I built felt natural. I do that for a living. The intro was like 45 seconds and took me like 45 minutes to film because it was acting and I don’t know how to do that
- learning about video editing features had an immediate payoff because video is so long
[0] I’m posting the videos at https://m.youtube.com/@bitlog-dev . I said if the first one got to 100 I’d commit to making at least 10, and I just crossed that threshold
He recruited for wall street trading firms; based on the finance types I've met over the years in NYC I would 100% believe some did this just because they hated losing. He was just making the point that you should never lie about your salary history, because he can help you if you didn't want to give it but he couldn't help you if you BSed everyone and got caught.
An old neighbor of mine was a headhunter. He once told me that some companies had a trick to get around the law. Upon getting hired, you'd sign a document saying that you'd agree to all policies in the employee handbook. Pretty standard stuff. One of the company policies was that you needed to prove any previous salary you stated in the negotiation. If it was too far off, they'd just terminate you. The trick is that they didn't ask at all during the hiring process; you're already hired and onboarded and then HR puts a meeting on your calendar to explain the policy to you.
Fun fact: I was on the Google Docs team from 2010-2015. Save didn't do anything but we still hooked up an impression to the keystroke to measure how often people tried to save. It was one of the top things people did in the app at first; it was comparable to how often people would bold and unbold text. And then as people gained confidence it went down over time.
There are some rose-colored glasses when people say this.
Programs didn’t auto save and regularly crashed. It was extremely common to hear someone talk about losing hours of work. Computers regularly blue screened at random. Device drivers weren’t isolated from the kernel so you could easily buy a dongle or something that single-handedly destabilized your system. Viruses regularly brought the white-collar economy to its knees. Computer games that were just starting to come online and be collaborative didn’t do any validation of what the client sent it (this is true sometimes now, but it was the rule back then).
I’m on tirzepatide and it’s crazy how it can truly reform habits. I’ve been a night snacker my whole life. I don’t even think about food after dinner anymore. At a bar, I used to pound down the last quarter of a beer so I could go get another. Now I might forget to finish it and I probably won’t get another. Now I feel full while eating for the first time in my life. It’s going to be truly transformative at scale, even knowing it doesn’t work for everyone.
Ex-Google, ex-Etsy
Writing: https://www.bitlog.com, https://www.clientserver.dev Email: [email protected] Twitter: @jakevoytko