Well, unlike a human, I cannot expect any these LLMs to take any ownership of the work they do. I cannot expect any given model and version (sonnet 4.6) to learn, improve and adapt over time. I cannot expect it's limitations to ever go away at the model level. So it is not like a human in most ways that I actually care about.
That said, I can't wait for LLMs to stop being AI and start being just another tool. Anything cursed with the "AI" label seems to go through this mess. In the earlier AI cycles, rules engines were considered "human-ish" and got hyped up, but today we just see then as just another tool available to us, and we're better off for it.
Agreed. Recently I was discussing the same point with a non-technical friend who was explaining that his CTO had decided to move from Digital Ocean to AWS, after DO experienced some outage. Apparently the CEO is furious at him and has assumed that DO are the worst service provider because their services were down for almost an entire business day. The CTO probably knows that AWS could also fail in a similar fashion, but by moving to AWS it becomes more or less an Act of God type of situation and he can wash his hands of it.
If you have a couple of hours to spare, I recommend listening to the episode of the Songhai Empire by the Fall of Civilizations podcast. They go into the fascinating historical background of the Timbuktu scholars and libraries. Link: <https://www.youtube.com/watch?v=GfUT6LhBBYs>
Actually it's pretty weird. This is a sports blog and content engine. And the author is someone who started this publication. I'm not sure what the motivation/context is at all here.
I've generally had a little bit more success with mocking when I'm hiding that dependency behind my own interface. So for example in Java, instead of trying to mock the AWS provided class, I write my own class (like a facade or repository pattern) which has a very simple interface of a success case and maybe a couple of relevant failure cases. It's calling the AWS library within it. But my mocks are at the level of my facade class which I find easier. The drawback is I'm not sure if there's a good general strategy to test the implementation of that facade. Most of the times the implementation is simple enough that I can do some simple integration tests for the most relevant cases, but there's always a risk that I am missing out some weird edge cases and I don't know how to properly deal with that.
That said, I can't wait for LLMs to stop being AI and start being just another tool. Anything cursed with the "AI" label seems to go through this mess. In the earlier AI cycles, rules engines were considered "human-ish" and got hyped up, but today we just see then as just another tool available to us, and we're better off for it.