> if you make the same argument for flight it looks really weak.
flight is an extremely straightforward concept based in relatively simple physics where the majority of the critical, foundational ideas involved were already near-completely understood in the late 1700s.
wouldn't call myself a detractor. i wouldn't call it a belief system i hold (i am an engineer 20 years into my career and would love to automate away the tedious parts of my job i've done a thousand times) as it is a position i hold based on the evidence i've seen in front of me.
i constantly hear that companies are running with "50% of their code written by AI!" but i've yet to meet an engineer who says they've personally seen this. i've met a few who say they see it through internal reporting, though it's not the case on their team. this is me personally! i'm not saying these people don't exist. i've heard it much more from senior leadership types i've met in the field - directors, vps, c-suite, so on.
i constantly hear that AI can do x, y, or z, but no matter how many people i talk to or how much i or my team works towards those goals, it doesn't really materialize. i can accept that i may be too stupid (though i'd argue that if that's the problem, the AI isn't as good as claimed) but i work with some brilliant people and if they can't see results, that means something to me.
i see people deploying the tool at my workplace, and recently had to deal with a situation where leadership was wondering why one of our top performers had slowed down substantially and gotten worse, only to find that the timeline exactly aligned with them switching to cursor as their IDE.
i read papers - lots of papers - and articles about both positive and negative assertions about LLMs and their applicability in the field. i don't feel like i've seen compelling evidence in research not done by the foundation model companies that supports the theory this is working well. i've seen lots of very valid and concerning discoveries reported by the foundation model companies, themselves!
there are many places in the world i am a hardliner on no generative AI and i'll be open about that - i don't want it in entertainment, certainly not in music, and god help me if i pick up the phone and call a company and an agent picks up.
for my job? i'm very open to it. i know the value i provide above what the technology could theoretically provide, i've written enough boilerplate and the same algorithms and approaches for years to prove to myself i can do it. if i can be as productive with less work, or more productive with the same work? bring it on. i am not worried about it taking my job. i would love it to fulfill its promise.
i will say, however, that it is starting to feel telling that when i lay out any sort of reasoned thought on the issue that (hopefully) exposes my assumptions, biases, and experiences, i largely get vague, vibes-based answers, unsourced statistics, and responses that heavily carry the implication that i'm unwilling to be convinced or being dogmatic. i very rarely get thoughtful responses, or actual engagement with the issues, concerns, or patterns i write about. oftentimes refutations of my concerns or issues with the tech are framed as an attack on my willingness to use or accept it, rather than a discussion of the technology on its merits.
while that isn't everything, i think it says something about the current state of discussion around the technology.
so to clarify your case, you are having it generate a new application, from scratch, and then benchmarking the quality of the output and how fast it got to the solution you were seeking?
i will concede that in this arena, there does seem to be meaningful improvement.
i said this in one of my comments in this thread, but the place i routinely see the most improvement in output from LLMs (and find they perform best) for code generation is in green field projects, particularly ones whose development starts with an agent. some facts that make me side-eye this result (not yours in particular, just any benchmark that follows this model):
- the codebase, as long as a single agent and model are working on it, is probably suited to that model's biases and thus implicitly easier for it to work in and "understand."
- the codebase is likely relatively contained and simple.
- the codebase probably doesn't cross domains or require specialized knowledge of services or APIs that aren't already well-documented on the internet or weren't built by the tool.
these are definitely assumptions, but i'm fairly confident in their accuracy.
one of the key issues i've had approaching these agents is that all my "start with an LLM and continue" projects actually start incredibly impressively! i was pretty astounded even on the first version of claude code - i had claude building a service, web management interface AND react native app, in concert, to build an entire end to end application. it was great! early iteration was fast, particularly in the "mess around and find out what happens" phase of development.
where it collapsed, however, was when the codebase got really big, and when i started getting very opinionated about outcomes. my claude.md file grew and grew and seemed to enforce less and less behavior, and claude became less and less likely to successfully refactor or reuse code. this also tracks with my general understanding of what an LLM may be good or bad at - it can only hold so much context, and only as textual examples, not very effectively as concepts or mental models. this ultimately limits its ability to reason about complex architecture. it rapidly became faster for me to just make the changes i envisioned, and then claude became more of a refactoring tool that i very narrowly applied when i was too lazy to do the text wrangling myself.
i do believe that for rapid prototyping - particularly the case of "product manager trying to experiment and figure out some UX" - these tools will likely be invaluable, if they can remain cost effective.
the idea that i can use this, regularly, in the world of "things i do in my day-to-day job" seems a lot more far fetched, and i don't feel like the models have gotten meaningfully better at accomplishing those tasks. there's one notable exception of "explaining focused areas of the code", or as a turbo-charged grep that finds the area in the codebase where a given thing happens. i'd say that the roughly 60-70% success rate i see in those tasks is still a massive time savings to me because it focuses me on the right thing and my brain can fill in the rest of the gaps by reading the code. still, i wouldn't say its track record is phenomenal, nor do i feel like the progress has been particularly quick. it's been small, incremental improvements over a long period of time.
i don't doubt you've seen an improvement in this case (which is, as you admit, a benchmark) but it seems like LLMs keep performing better on benchmarks but that result isn't, as far as i can see, translating into improved performance on the day-to-day of building things or accomplishing real-world tasks. specifically in the case of GPT5, where this started, i have heard very little if any feedback on what it's better at that doesn't amount to "some things that i don't do." it is perfectly reasonable to respond to me that GPT5 is a unique flop, and other model iterations aren't as bad, in that case. i accept this is one specific product from one specific company - but i personally don't feel like i'm seeing meaningful evidence to support that assertion.
you didn't provide an anecdote. you just said "it's better." an anecdote would be "claude 4 failed in x way, and claude 4.5 succeeds consistently." "it is better" is a statement of fact with literally no support.
the entire thrust of my statement was "i only hear nonspecific, vague vibes that it's better with literally no information to support that concept" and you replied with two nonspecific, vague vibes. sorry i don't find that compelling.
"troubling" is a wild word to use in this scenario.
"it's better at coding" is not useful information, sorry. i'd love to hear tangible ways it's actually better. does it still succumb to coding itself in circles, taking multiple dependencies to accomplish the same task, applying inconsistent, outdated, or non-idiomatic patterns for your codebase? has compliance with claude.md files and the like actually improved? what is the round trip time like on these improvements - do you have to have a long conversation to arrive at a simple result? does it still talk itself into loops where it keeps solving and unsolving the same problems? when you ask it to work through a complex refactor, does it still just randomly give up somewhere in the middle and decide there's nothing left to do? does it still sometimes attempt to run processes that aren't self-terminating to monitor their output and hang for upwards of ten minutes?
my experience with claude and its ilk are that they are insanely impressive in greenfield projects and collapse in legacy codebases quickly. they can be a force multiplier in the hands of someone who actually knows what they're doing, i think, but the evidence of that even is pretty shaky: https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...
the pitch that "if i describe the task perfectly in absolute detail it will accomplish it correctly 80% of the time" doesn't appeal to me as a particularly compelling justification for the level of investment we're seeing. actually writing the code is the simplest part of my job. if i've done all the thinking already, i can just write the code. there's very little need for me to then filter that through a computer with an overly-verbose description of what i want.
as for your search results issue: i don't entirely disagree that google is unusable, but having switched to kagi... again, i'm not sure the order of magnitude of complexity of searching via an LLM is justified? maybe i'm just old, but i like a list of documents presented without much editorializing. google has been a user-hostile product for a long time, and its particularly recent quality collapse has been well-documented, but this seems a lot more a story of "a tool we relied on has gotten measurably worse" and not a story of "this tool is meaningfully better at accomplishing the same task." i'll hand it to chatgpt/claude that they are about as effective as google was at directing me to the right thing circa a decade ago, when it was still a functional product - but that brings me back to the point that "man, this is a lot of investment and expense to arrive at the same result way more indirectly."
every time i say "the tech seems to be stagnating" or "this model seems worse" based on my observations i get this response. "well, it's better for other use cases." i have even heard people say "this is worse for the things i use it for, but i know it's better for things i don't use it for."
i have yet to hear anyone seriously explain to me a single real-world thing that GPT5 is better at with any sort of evidence (or even anecdote!) i've seen benchmarks! but i cannot point to a single person who seems to think that they are accomplishing real-world tasks with GPT5 better than they were with GPT4.
the few cases i have heard that venture near that ask may be moderately intriguing, but don't seem to justify the overall cost of building and running the model, even if there have been marginal or perhaps even impressive leaps in very narrow use cases. one of the core features of LLMs is they are allegedly general-purpose. i don't know that i really believe a company is worth billions if they take their flagship product that can write sentences, generate a plan, follow instructions and do math and they are constantly making it moderately better at writing sentences, or following instructions, or coming up with a plan and it consequently forgets how to do math, or becomes belligerent, or sycophantic, or what have you.
to me, as a user with a broad range of use cases (internet search, text manipulation, deep research, writing code) i haven't seen many meaningful increases in quality of task execution in a very, very long time. this tracks with my understanding of transformer models, as they don't work in a way that suggests to me that they COULD be good at executing tasks. this is why i'm always so skeptical of people saying "the big breakthrough is coming." transformer models seem self-limiting by merit of how they are designed. there are features of thought they simply lack, and while i accept there's probably nobody who fully understands how they work, i also think at this point we can safely say there is no superintelligence in there to eke out and we're at the margins of their performance.
the entire pitch behind GPT and OpenAI in general is that these are broadly applicable, dare-i-say near-AGI models that can be used by every human as an assistant to solve all their problems and can be prompted with simple, natural language english. if they can only be good at a few things at a time and require extensive prompt engineering to bully into consistent behavior, we've just created a non-deterministic programming language, a thing precisely nobody wants.
totally familiar with that - more often than not (in that it's always been the case) that is a function of how the company chose to build things, however, and not the base framework the product was built on.
in fact, the greatest source of that kind of trouble, in my experience, is constantly changing approaches and paradigms and patterns throughout the codebase to do something "more modern" without fully committing, leading to a super stratified codebase with dozens of patterns and weird hooks and bindings to make them all get along...
realistically i've worked at very few companies whose delivery is held back meaningfully by the framework something is built in.
when there's friction, it's much more likely to come from poor planning, or constantly adding more functionality without stopping to reconsider architecture, or one of a thousand more organizational issues.
the innovation delivered by basically anyone working in software is extremely rarely a function of the tools they use to build the software, and many extremely successful products effectively started as CRUD apps, just organized in a way that really served a specific use case well.
the stuff i recall that truly transformed the way i've experienced the web - (what was at the time) AJAX, webGL, the canvas tag, websockets - tend to be shipped in the browser, and function equally well in basically any framework. i don't really think that i can point to a single new framework that truly changed the way i experience the web meaningfully.
react is probably the closest i can recall, but primarily because it was the one that caught on and made building really rich SPAs fashionable after the long slushy period of knockout and angular and backbone and handlebars and the thousand other disparate things cobbled together by every company. it catching on and taking over most of the industry meant people could move between jobs easier, contribute more quickly, and take easier advantage of countless libraries because they were either natively made for react or there was plenty of documentation and support for integrating them.
having that broad a universe of support might actually be a main source of innovation, when you think about it. having it be effortless to integrate basically anything in the js universe into your project because it's well-documented and has been done a thousand times means you can focus more easily on the unique parts of your project.
i'm definitely a little jaded, and 20ish years into my career i'm more business-minded than i was when i started, but i struggle to imagine a framework so profoundly and uniquely enabling of new things, that would have such a meaningful impact on my bottom line, that i would choose it and the trouble of hiring experienced engineers comfortable with it (or training new ones) when i could just bring on literally anyone in the entire industry, because basically all front-end devs are comfortable in react.
at what point does it make sense to say “maybe you don’t deserve the benefit of the doubt” because it sincerely feels well past that point by all measures.
i needed an "oh, that's really nice" story today. this delivered.
in every way, this seems well-intentioned, quirky, cute, fun, and positive. unless there's some subtext i'm missing, this is just a good and nice thing happening that's great for everyone involved.
apple ebbs and flows in terms of how on the ball they are in any given area, but it feels we're at a strange inflection point where their hardware is the best it's ever been and the software is inexplicably in a death spiral
i've been a heavy safari user for a while, mainly because i do make extensive use of the tab and history syncing across all my devices, and safari is the only actual browser you get on iOS - might as well use the native version.
lately safari has this habit of, on some websites, entirely locking up my device while loading web pages. like full on hard lock can't switch windows, nothing can be done, sometimes for upwards of 30 seconds. to go to my electric company's website, i have to use chrome. otherwise my computer becomes unusable.
i am not suggesting that their website isn't awful (it is) but it is inexcusable that on an M2 max laptop with 64 GB of ram that loading a slow or bulky website should make my computer completely unusable. i do not understand how this hasn't been addressed. it was intermittent before but it's a daily occurrence now.
this along with all the weird visual glitches, notifications snapping between sharp-edged boxes and rounded boxes repeatedly, sudden drops in frame rate on my iPhone display that seem to start and end for no reason, and it's starting to feel like everyone at apple uses their devices as beautiful paperweights primarily and doesn't actually interact with the software at all...
the thing that frustrates me deeply is i've explored the android ecosystem extensively (i've owned several samsung and pixel devices, even very recent ones as second phones) and find that whole space even worse and more unpleasant, with the shovelware play store and a general unpleasant and janky UI that has never felt right to me. so it's like... what's the GOOD option now?
the way i've heard it explained is functionally that the ultra rich are either leaning towards things like those private suites onboard a large plane, or flying in a private jet.
people don't mind the experience of flying in a plane or the time it takes for the most part - they mind being uncomfortably crammed into a seat for hours on end with another person spilling into their lap in a loud, stuffy cabin. otherwise, it's just hanging out in a different place than you usually do.
at the point you're paying for a resort hotel room with a shower, bed, privacy, internet and a tv in the air... who cares if you spend a few extra hours? the only example of a supersonic airliner that i can point to, the concorde, was actually fairly uncomfortable and cramped because of the way it was designed. it's likely (though i've been wrong before) that future supersonic planes would make similar tradeoffs to try and minimize weight and drag and maximize fuel economy - you will trade comfort for speed.
i think most of the people you're talking about would prefer 8 hours in a private hotel room (or full on private jet) with a full bar, bottle service, a shower and fancy meals to 2-3 hours cramped in a relatively small cabin after the novelty wears off. given how much easier it is to effectively meet across the ocean without traveling, the market for ultra-fast flights to get a one-day trip over with is also likely smaller.
i can't say i know any of these facts for certain, but previously when discussing the return of supersonic flights with folks who know better than i, this was the general sentiment. it makes reasonable sense to me on its face.
i don't. i'm explicitly choosing not to be pedantic and instead hoping you'll take what i say as what it obviously is intended to mean and not as a very specific and accurate phrasing to be disassembled and torn apart without acknowledging the overall intent of the message.
as a person who likes airplanes (and airliners in particular,) i think it's cool that a commercially-focused aircraft manufacturer has managed to return to a type of flight that has primarily been relegated to military operations for a very long time
today i am not thinking any further ahead than "wow, they did a really cool thing and made a supersonic test platform for a commercial airliner."
there will be lots of future questions and concerns but we are far off from them, because they are not even close to scaling this up and there are so many gaping holes in the plan that i don't take it seriously at the moment.
commercial and private jets generally cap out around mach 0.9
i am very rusty on the economics and details of supersonic commercial flight, but the general gist as i recall is:
- going much faster scales up the cost of flying at a rate that's hard to justify for how much time it saves. there is less case in the 2000s for "having to be in london in 3 hours from NY" than there previously was, too.
- noise restrictions and such limit the usefulness of planes that are set up to fly that fast as people don't like being underneath constant sonic booms, so the routes that supersonic passenger flights were relegated to are mostly over water.
it is just way cheaper and easier to fly subsonic, and if you're on a private jet anyway it's not like you're uncomfortable while traveling.
there hasn't been supersonic civil aviation, as far as i am aware, since the concorde was grounded. there are no active commercial aircraft capable of going supersonic.
this is significant because it's the first civil aircraft to reach that milestone since the ending of the concorde program.
wow! this sure is great! gemini has worked so great up until this point - for example, i learned that a man who died in 1850 is one of three private owners of the airbus a340-600 last week! i'm so glad gemini exists and i absolutely cannot wait to experience a world wherein people get news from it.
flight is an extremely straightforward concept based in relatively simple physics where the majority of the critical, foundational ideas involved were already near-completely understood in the late 1700s.
i really don't think it's fair to compare the two