"A sufficiently precise spec is code". I've read somewhere here before.
So guardrails, i.e. sufficiently precise spec and tests, will need to be as strict as the LLM is bad at getting the right context and asking back the right questions. I suppose at that point not much difference between a human engineer and it.
I know this tool was meant for AI, but I am more interested in using it myself when exploring new code bases or even my own, when I want to refactor something and want an overview of where to change stuff. LSPs do that, but this tool sounds like it can go one step further.
My current understanding is that regular consumer GPUs are very much capable if they had enough VRAM, and buying like 2 isn't out of this world.
A 64GB consumer GPU not existing has no technical nor financial (buyer side) reason to it, other than market segregation. If a 24GB one costs 1000 bucks, people are ready to pay quadruple that for a 32GB GPU. And VRAM is the main bottleneck for running larger LLMs, not performance. And some resort to offloading to RAM etc...
Segregation between pro and consumer hardware are in a big part artificial I think, for fatter margins.
Otherwise, yeah Hyperscalers have deep pockets, but they can only have it by getting back money from their users/customers, and we're gonna give them that by using their products and paying for it (indirectly or directly).
I don't think it's a good thing that only select companies gatekeep AI and it feels to me like it's going that way with chip prices.
How about simply making the website an app and have it load your makedown file with a button and file browser. Just like e.g. https://app.diagrams.net/
And I believe you can then tell the browser that you need no network communication at that point. And a user can double check that.
So guardrails, i.e. sufficiently precise spec and tests, will need to be as strict as the LLM is bad at getting the right context and asking back the right questions. I suppose at that point not much difference between a human engineer and it.