Pretty interface, although I remain unconvinced of how I'd actually use it. If I'm just prototyping for myself, LLM providers offer a decent history, and I rarely need to share notebook-style explorations of LLMs with my team. For production use cases at logicloop.com/ai we just add our prompts into code.
What's the use case you're envisioning people using AI notebooks for?
Appreciate the effort you put into this. In terms of user flow, I typically need one or two of these use cases at a time. In that case, I just type my conversion into a search engine like Google, and often use their default box.
Can you share what types of use cases you've seen people use KodyTools for?
Yup, that's part of it but I mean it bidirectionally - users can accidentally leak data to models too, which is concerning to SecOps teams without a way to monitor / auto-redact.
These common issues tend to prevent LLMs from being used in the wild:
* Data Leakage
* Hallucination
* Prompt Injection
* Toxicity
So yes it does include prompt injection, but is a bit broader. Data Leakage is one that several customers have called out, aka accidentally leakage PII to underlying models when asking them questions about your data.
I'm evaluating tools like Private AI, Arthur AI etc. but they're all fairly nascent.
Great starting point! These diagrams notably miss a LLM firewall layer, which is critical in practice to safe LLM adoption. Source: We work with thousands of users for logicloop.com/ai
Very cool, but curious if you see people actually directly interacting with LLMs vs in a script as part of a larger application? I see myself needing debugging, visualizing output etc. so much that an IDE makes more sense to me as an interface, so want to learn about cases where that doesn't.
Aw man, sorry to hear this about your friend. Inanimate objects are directly subject to the laws of physics, but living beings that have intention and will are able to circumvent those. For example, I can jump despite gravity existing. Yes, "in the long run, we're all dead", but applying laws of entropy as a reason to not live seems indicative of a lack of will, rather than a natural law every being must follow.
As long it's clear that this is fiction, how would something like this be more damaging than a series like The Man in the High Castle, or other sci-fi that imagines an alternate universe? I think it's a nifty technique that allows us to viscerally imagine and live out our parallel universes.
Yeah, in general the more data you're able to use (assuming the context window supports it), the better results tend to be. We arrived at the data schema being a good enough compromise at which the benefits outweigh the risks for several use cases. Besides, some data stores that are generated by third-parties actually have common schemas (think Sendgrid / Hubspot activity data), so you're not risking much but potentially gaining a lot of sales ops productivity.
This seems overall well-written and well-explained, but curious for that piece on fine-tuning. This article only recommends it as a last resort. That makes sense for a casual user, but if you're a company seriously using LLMs to provide services for your customers, wouldn't the cost of training data be offset by the potential gains you have and the edge cases you might automatically cover by fine-tuning instead of trying to whack-a-mole predict every single way the prompt can fail?
Thanks for sharing how you keep the discussion quality high here. We responded to some thought-provoking questions on this thread about differentiation vs ChatGPT, other SQL Copilots, edge cases like more complicated queries etc. that we believe other HN users and makers in this space will benefit from. As I responded in @chatmasta's vouch (thank you!), we're data and software engineering experts, and happy to provide our original perspective on any other questions you have for us. Thanks for your consideration!
On simpler multi-table joins we've been able to product good results, and we've done a lot of prompt engineering to make sure it takes the schema very seriously so that prevents hallucinations too. We're always finding new edge cases and fixing those as we go.
Yup, our bet is that people are going to want better integration into data sources and actions resulting from their queries, and that a lot of business value will come from that.
Appreciate the vouch, @chatmasta! I actually did write those comments myself, and think that the 3-point explanation makes for a clear explanation. My cofounder and I both started our careers as software engineers, and I've been in the data space for over a decade.
We use a combination of APIs from existing LLM providers, and do some serious prompt engineering to get the best from them. We're starting to train models on more SQL-specific prompts now.