I agree, real intelligence may also potentially be explained as all "math and probability", whether it's neurons or atoms. A key difference between our brains and LLMs is that the underlying math behind LLMs is still substantially more comprehensible to us, for now.
It's common to believe that we have a more mystical quality, a consciousness, due to a soul, or just being vastly more complex, but few can draw a line clearly.
That said, this article certainly gives a more accurate understanding of LLMs compared to thinking of them as if they had human-like intelligence, but I think it goes too far in insinuating that they'll always be limited due to being "just math".
On a side note, this article seems pretty obviously the product of AI generation, even if human edited, and I think it has lots of fluff, contrary to the name.
That's reasonable, I certainly believe that there are many fake and manipulative people who say what's best for their personal gain, perhaps even the majority. But I still think it's reasonable to imagine that there are some people are genuinely concerned about this.
I'm not highly concerned but I think there is merit in at least contemplating this problem. I believe that it would be better to reduce suffering in animals, but I am not vegan because the weight of my moral concern for animals does not outweigh my other priorities.
I believe that it doesn't really matter whether consciousness comes from electronics or cells. If something seems identical to what we consider consciousness, I will likely believe it's better to not make that thing suffer. Though ultimately it's still just a consideration balanced among other concerns.
When you put it like that, it makes me wonder if we can just stick to using the self-driving cars in the Bay Area and not go to these bad and dangerous places.
I agree with this, it reminds me of how most people don't need to write assembly anymore, but it still helps with certain projects to have that understanding of what's going on.
So some people do develop that deeper understanding, when it's helpful, and they go on to build great things. I don't see why this will be different with AI. Some will rely too much on AI and possibly create slop, and others will learn more deeply and get better results. This is not a new phenomenon.
This sounds like an assignment to learn to use LLMs, which as an isolated assignment sounds reasonable. Students should learn how to use tools of all kinds to maximize their effectiveness. It might be a bigger problem if all assignments are done like this but I doubt that's the case.
Even as AI generates more writing and code, we still have a way of ranking quality: Good writing and successful projects tend to get more popular and prominent. This selection can allow LLMs to continue to improve. They get a huge flow of slop, but they generate based on the patterns correlated with better quality. The model developers can also develop better ways to curate the input data themselves and keep the slop at bay.
It's not a guaranteed or trivial mechanism, but I don't think we need a new breakthrough either.
It really depends what I'm building, and I find that these are often additional tasks that are done later after the core functionality is validated. I've most often built apps used internally or to test concepts for early user feedback, that had a relatively low bar. But regardless, I can verify these without as much deep knowledge of the code by trying them myself, but can't verify easily that the backend was coded securely and properly.
But, covering all these cases and doing all the polish and animations expected of high quality frontends has usually taken much longer when I've needed to do it, 80% of the dev time has been frontend in some cases.
I'm saying from the perspective of someone overseeing a frontend dev, that I can just try out the app feature and see if things seem to be working as expected. Though as you mention, it's necessary to check a variety of devices and other edge cases, depending on the project requirements.
With backend though, even if it seems to work, there can be severe hidden problems with the architecture and security, so I really need to trust the backend dev or verify things deeply myself in order to ensure quality.
If I'm making a quick app for a startup, I can often hire relatively less experienced frontend devs, but have to care much more about the backend.
This isn't really true, everyone has a different basal metabolic rate, and effectiveness with absorbing calories from food can vary as well. Even small differences can add up to large effects, the difference between being at net-zero, or having caloric surplus or deficit every day.
That said, in practice it may be reasonable advice on average, but there's also a problem where it's not very practical to eat the "same" calories as someone else, unless they are together with you all the time.
I find that frontend takes most of the dev time for most apps, and I certainly consider it "harder" to get everything working to the quality level I want. However, backend work is usually more critical, as problems can result in data loss or security issues. Frontend problems often just result in bad UX, and they are easier to do a surface-level check too (just use the app and check that it works).
Due to this, companies may have a higher bar of expertise for backend which may give the impression that it is "harder", but I don't think this is a very important distinction.
Related to taking tiny steps, I've set up a daily habit checklist with the lowest bar possible, even lower than the author's suggested log statement. When it comes to software dev, it's just "open my IDE and look at my notes for what to do next". This usually just takes 10 seconds, but it's the first step in starting and usually leads to me doing at least a bit more, so it's helpful when I'm at my lowest in terms of energy. And even if I do nothing else, I get some satisfaction that I at least completed my to-do and did a tiny bit more than nothing for the day.
This seems like a case of selection bias, where they are looking at all the Gen AI startups and seeing that they are making revenue faster than previous startups. But Gen AI startups have mostly only started very recently, so it's obvious that all the successes must have grown fast, as they haven't been around long enough to grow slowly. Maybe in 5 years, we'll see a lot of cases of successful startups that took a slower growth trajectory instead.
But whether it's short-sighted for the investors or not, I think the takeaway for founders is "investors now expect you to make more revenue faster, and B2C applications are more interesting than before".
It's common to believe that we have a more mystical quality, a consciousness, due to a soul, or just being vastly more complex, but few can draw a line clearly.
That said, this article certainly gives a more accurate understanding of LLMs compared to thinking of them as if they had human-like intelligence, but I think it goes too far in insinuating that they'll always be limited due to being "just math".
On a side note, this article seems pretty obviously the product of AI generation, even if human edited, and I think it has lots of fluff, contrary to the name.