300+ Gb in VRAM, If it is as good as opus 4.7 , would be beneficial to small orgs, question is what is the cost operating this and at what price point does it help an engineering team.
I used to run a similar company in this space. gearsec.io
My 2 cents are that no one we spoke was comfortable with us having mitm plus access to their secrets no matter how much we told we'll host it in your cloud.
The few who agreed were rigorously testing our product and asked for code SBOMs before even a pilot.
Infiscals agent vault might be the best middle ground for this kind of setups I feel sometimes
I personally consult LLMs over technical stuff. But for the more nuanced "human" side of things, I feel it gives you answers from both sides
For instance, I wanted to know if it's better to setup a call with a client in a coffee shop over their office. Almost all LLMs will give you whatever answer you want based on how much stress you put on one particular answer.
I'm not saying that traditional search engines will show you only one sided results, but you'll see reddit threads that will have a bit of conversation before arriving at some conclusion. You'd read it and go hmm this question had XYZ setup and they picked the coffee shop as a great place. My own situation is perhaps (for sake of argument) not XYZ, so it's better to have the meeting in the client office.
This is a community driven decision where I have certain basis if I want to revisit my decision.
With an llm I find that it will happily just give you one of the two options and keep switching in the middle if you ask it to. That's what makes me a bit a skeptical.
With technical stuff, there are direct solutions that fit your usecase and you can just use it and move on.
one of the biggest hurdles I see still, is that the red tape around this is so massive, it takes away a lot of the valuable time of actually working on something for your startup.
Take me for example, I spend nearly 3 4 days worth of efforts each month following up on different websites, and with agents of so called "compliance management companies"
I just hope they did something that solves this and makes it very easy to do corporate compliance.