i think in the long term the problem is going to be this: a great small model always come from a greater large model, but the larger base model keep getting larger and more closed sourced
either ai closes the loop which is unlikely or ai is not good and a small group of people manage to resist the urge to talk to it and take over everyone that does eventually
i feel like we are using ai to solve virus detection in many cases, and in theory this is the same complexity as the halting problem.
evolutionary search is better than hard coded algorithms at finding solutions to np problems and this is similar to that. ai will be better security engineers than humans.
yeah anthropic tries to address this through mechanistic interpretation but not sure they are progressing as fast in that domain as their model development
so long as there's no algorithm breakthrough