Possibly overly cynical, but I think a lot of people who are getting forced to shoehorn LLMs into their applications don’t have a choice about NOT using an LLM for the required use case.
Testing and validation kind of imply that it won’t ship if it isn’t fit for purpose, which isn’t an option and most devs will know it’s kind of junk already.
That’s not to say there aren’t good use cases, but when forced to add an LLM somewhere it doesn’t work, and you have no examples of “correct” output anyway, validation is usually an afterthought.
Ugh I am dealing with an amazingly productive platform team who churn out so much stuff that the product teams just can’t keep on top of all the changes to tools and tech.
That one team is super productive at the cost of everyone else grinding to a halt.
I have to work with a DBA who has decided that nothing new gets developed using Postgres and is confident that our use cases are best served with a document db… all without knowing any of our use cases, requirements or constraints.
Now I just don’t involve him in anything unless forced.
Honestly this is a concern for me in a non boogeyman way. I joined a company to work on their edtech product for kids and got assigned to work on their AI product. I have no idea how to be confident and prove that the gen ai won’t tell the kids harmful things. We can try all kinds of things but I don’t know how to PROVE it won’t.
Testing and validation kind of imply that it won’t ship if it isn’t fit for purpose, which isn’t an option and most devs will know it’s kind of junk already.
That’s not to say there aren’t good use cases, but when forced to add an LLM somewhere it doesn’t work, and you have no examples of “correct” output anyway, validation is usually an afterthought.