I thought distillation meant small models don't have to compete with the big models and can always eventually achieve close parity, but it's just a matter of time to do the distillation? (i.e. how much lag do you want to live with) Am I oversimplifying?
The reason why you will never get software engineers (in companies) to accept the man behind the curtain is liability. If a human software engineer is still responsible for what happens when the AI developed code has a catastrophic bug or security vulnerability, then the only way for the human to know if there is a problem is to be able to read through the code or run it through some <insert advanced formal verification tool here> that guarantees zero issues.
I think we eventually end up at the tool approach via vendors providing the tools to other companies, but it still feels like there's a long road ahead to get there.
For sure Freud and modern therapy as practiced today are very different. Freud influenced the language, structure, and goals of talk therapy more than the exact methods most therapists use now. Modern therapy kept many of his clinical observations about inner conflict and relationships, but dropped or revised a lot of his more speculative theory.
Since mental illness or other trauma is entirely contained to the brain, you can in fact say that the problem is entirely biological. We are starting to see the tech industry make real inroads to biology. Neuralink, gene therapies, AI designed drugs, etc. All of these innovations will decrease the need for therapy, which at best you can say helps people learn to live with conditions, but never permanently fixes the problem.
People have been doing self-examination for a long time, but Freud's use of psychoanalysis is a fairly modern phenomenon and it's benefits are dubious. Modern therapy looks increasingly like pseudoscience. I expect biotech/AI advancements to make much of modern therapy irrelevant over time, as we obtain fine-grained control over the actual processes in the brain causing various afflictions.
Introspection is the conscious examination of one’s own mental, emotional, and cognitive processes to improve self-awareness. I think Marc's critique here is a lot of what can be learned about past mistakes is outside of an individual's own failings.
I was recently reading a post about how the Claude Code leak and Boris Cherny had the following to say..
"Mistakes happen. As a team, the important thing is to recognize it’s never an individuals’s fault — it’s the process, the culture, or the infra.
In this case, there was a manual deploy step that should have been better automated. Our team has made a few improvements to the automation for next time, a couple more on the way."
When complex systems fail often there is more than one thing that went wrong. Uncovering what those things are is important, so that you can address them and prevent them from happening again. Once fixed, it is on to the next task and no need to dwell on the past.
This blog post and all the comments in response feel very tautological. I think Marc has a fairly simple point here, which is don't spend time dwelling on the past. Learn from the past, take away information about how things can be improved, but then make a plan (for whatever it is that you are building/doing) and move forward with that plan.
In the podcast, he basically lays out that the A16Z thesis is that there is not enough technology, information, and intelligence in the world, so they are going out and investing in companies/ideas that can make an impact in these areas. That requires learning from the past, but not dwelling on it. Seems like a very sensible and positive approach to me.
> laws are only as good as we have systems in place that are willing and ABLE to enforce them.
The 'able' part is the critical insight. Laws are too often passed that really have no ability to be enforced, but end up adding bureaucratic processes that law abiding companies have to follow. This also implies that governments need to actively clean up existing laws, which almost never happens unless there is enough support to pass a new law to actively supplant the old one.
Okay, so what's the best solution? What's even just a better solution than Docker? I mean really truly lay out all the details here or link to a blog post that describes in excruciating detail how they shipped a web application and maintained it for years and was less work than Docker containers. Just saying "a far far simpler solution is to just link statically or ship dependencies adjacent to the binary" is ignoring huge swaths of the SDLC. Anyone can cast stones, very few can actually implement a better solution. Bring the receipts.
We know there is a real problem, awareness is not the issue. (I've been aware of it since the mid 90's) It is ignored by large industries and governments. The incessant pounding of the useless drum of individual action continues to go absolutely nowhere. We need government and industry to take action not individuals. I will no longer placate this idea that individual action is at all useful.
That is short hand. The problem exists of course, but it is improbable that it will actually occur in our lifetimes. An asteroid could slam into the earth or a gamma ray burst could sanitize the planet of all life. We could also experience nuclear war. These are problems that exist, yet we all just blissfully go on about our lives, b/c there is basically nothing that can be done to stop these things if they do happen and they likely won't. Basically we should only worry about these problems in so much as we as a species are able to actually do something about them.
This happens even today. If a knowledgeable person leaves a company and no KT (or more likely, poor KT) takes place, then there will be no one left to understand how certain systems work. This means the company will have to have a new developer go in and study the code and then deduce how it works. In our new LLM world, the developer could even have an LLM construct an overview for him/her to come up to speed more quickly.
Most people have no idea how to hunt, make a fire, or grow food. If all grocery stores and restaurants run out of food for a long enough time people will starve. This isn't a problem in practice though, because there are so many grocery stores and restaurants and supply chains source from multiple areas that the redundant and decentralized nature makes it not a problem. Thus it is the same with making your own food. Eventually if you have enough robots or food replicators around knowing how to make food becomes irrelevant, because you always will be able to find one even if yours is broken. (Note: we are not there yet)