Show HN: Roboco-op, a computational blackboard for efficient human/AI collab(observablehq.com)
observablehq.com
Show HN: Roboco-op, a computational blackboard for efficient human/AI collab
https://observablehq.com/@tomlarkworthy/robocoop
31 comments
Once you start editing AI responses you realize how limited and imprecise our UX so far has been. It's like a parent correcting a child during development. It's a huge help in getting it solving hard tasks. You can quickly transfer what you know and then let it generate from there. Huge win.
Stay out of trouble!
I guess I'm confused as to what I'm seeing. It appears to an LLM in a Jupiter notebook-esque environment. And that you can modify the LLMs output?
Is there more to it than that?
Is there more to it than that?
You ask a question. It outputs a cell. The cell contains the the answer and the prompt. There is no state, unlike the webapp. When constructing an API call, we use the materialized cells to reconstruct the context. Because of the notebook format, it's easy to edit everything. Because the LLM is speaking code, it's easy to check everything. You can feed the cell value outputs back into the prompt.
Cells can depend on each other, so you can build up. Cells can be prose, so you can plan. You can change the plan halfway. You can edit the AIs plan. You can forget the plan and do something else without affecting what happened after. You can keep the dialogue and context tight and information dense.
When generating a response, the LLM considers it's previous responses, which you can edit. So you get very fine control over its chain-of-thought. No need to use system prompt to train.
It's simple, but the emergent properties enable much more powerful collaboration.
Cells can depend on each other, so you can build up. Cells can be prose, so you can plan. You can change the plan halfway. You can edit the AIs plan. You can forget the plan and do something else without affecting what happened after. You can keep the dialogue and context tight and information dense.
When generating a response, the LLM considers it's previous responses, which you can edit. So you get very fine control over its chain-of-thought. No need to use system prompt to train.
It's simple, but the emergent properties enable much more powerful collaboration.
This makes sense and tells me what the benefits are in an explicit manner. Thank you.
First impression, oh my, er, what is this, deeper reflection, wow, so I can sort of program this like a smart spreadsheet where cells can contain LLM results which can feed into LLM prompts, and as I alter the contents of a cell, or LLM provides new results, the whole thing is 'recalculated', did I get that about right? Really looking forward to seeing how the UI develops in this space, so many ideas to explore.
Exactly. It needs to be experienced to understand the effect on workflow. I don't really know how to use it optimally yet, but it has very quickly learnt observable idioms without a hefty initial prompt, which it never normally can.
The cells become manipulatable knowledge memes. That they are computer checked and editable totally changes the speed you can teach by example, and therefore how quickly you can upskill it to do what you actually want it to do.
The cells become manipulatable knowledge memes. That they are computer checked and editable totally changes the speed you can teach by example, and therefore how quickly you can upskill it to do what you actually want it to do.
Mashing this up with something like https://scratch.mit.edu/ would float my boat :)
Take a look at Iron Clad. It's kind of like you describe. Not quite, but close.
Thanks, https://github.com/Ironclad/rivet looks interesting, langchain vibes, will take a look :)
Intriguing but a simpler example would probably have gotten the idea across
much more effectively
[deleted]
That is pretty brilliant. I’m just trying to get my head around the power of chaining all this together.
[deleted]
Unrelated in all but name, but for those who enjoy the robocop movies and are gamers, the new game (delta city) feels very true to the movies. It's not really a challenge, and the gameplay is pretty linear, but it's an enjoyable ~12hrs if you want to be a bullet sponge cleaning up the criminal slime.
[deleted]
Have you experimented with sending back the evaluated result in some way? How often does it give back broken code?
The block 'Cell Values to Include' let's you forward values selectively. The v0 had auto feedback but it wasn't robust, quite risky on credits, and not that useful in most cases, so now it's a multi select per prompt opt in.
This could have done with a slightly gentler and clearer introduction.
Goes from something was trained by ai to markdown to visualisations to react stuff.
AI and notebooks sounds like it could be a powerful concept - just can't tell what's going on here in this article.
Which parts of this is AI generated and what's the user input. maybe I'm looking at llm output? Though the cells seem to contain input and output combined with && somehow. Maybe that's the llm echoing back the input? Is the ai generating the cells or interpreting them? Both?
Goes from something was trained by ai to markdown to visualisations to react stuff.
AI and notebooks sounds like it could be a powerful concept - just can't tell what's going on here in this article.
Which parts of this is AI generated and what's the user input. maybe I'm looking at llm output? Though the cells seem to contain input and output combined with && somehow. Maybe that's the llm echoing back the input? Is the ai generating the cells or interpreting them? Both?
I agree but I found people bounce with big intros, so it's a bad attempt at "show don't tell."
The cells were written by AI, but the metadata data that generated the cell is mixed in (thats the {...} && bit).
So now the full chat context can be derived from the notebook. Because the notebook is editable, you can fully edit the chat context in an ergonomic way. Both the prompt, the AI code output. So when the AI makes a mistake (it often does), you can correct it. Your ability to detect mistakes is accelerated by the fact it's a computation based conversation. Because the chat context is derived from the editable state, your corrections are internalised into the chat conversation. It continues as if it answered that for real.
So in the notebook you see that the AI is rapidly become an Observable notebook expert, because I have manually coarse corrected its chain-of-thought. An out-of-the box chatGPT session cannot do what is in that notebook. My expertise have been transferred. Once it knows how to solve something in the chat context, its very easy for it to repeat the pattern for new situations. Having it as a standalone forkable self-enclosed notebook artifact is useful. The existing notebook infra allows you to copy and paste cells across notebook too. So I am thinking we can curate stand alone skill memes. The examples are all over the shop just to see what kinda of things I could get it to solve, I imagine a real use would concentrate on upskilling just enough to solve a specific class of problem.
The cells were written by AI, but the metadata data that generated the cell is mixed in (thats the {...} && bit).
So now the full chat context can be derived from the notebook. Because the notebook is editable, you can fully edit the chat context in an ergonomic way. Both the prompt, the AI code output. So when the AI makes a mistake (it often does), you can correct it. Your ability to detect mistakes is accelerated by the fact it's a computation based conversation. Because the chat context is derived from the editable state, your corrections are internalised into the chat conversation. It continues as if it answered that for real.
So in the notebook you see that the AI is rapidly become an Observable notebook expert, because I have manually coarse corrected its chain-of-thought. An out-of-the box chatGPT session cannot do what is in that notebook. My expertise have been transferred. Once it knows how to solve something in the chat context, its very easy for it to repeat the pattern for new situations. Having it as a standalone forkable self-enclosed notebook artifact is useful. The existing notebook infra allows you to copy and paste cells across notebook too. So I am thinking we can curate stand alone skill memes. The examples are all over the shop just to see what kinda of things I could get it to solve, I imagine a real use would concentrate on upskilling just enough to solve a specific class of problem.
> Because the chat context is derived from the editable state, your corrections are internalised into the chat conversation.
Ah right. That makes more sense to me. Thanks for explaining
Ah right. That makes more sense to me. Thanks for explaining
As its quite a steep learning curve I will add announce better instructions and UX upgrades here https://www.reddit.com/r/robocoop/
I very much believe in this direction both as a practicing data scientist, and working daily with folks who kinda know SQL or Python but are really more experts in other stuff, like security, k8s, supply chain, etc.
For a slightly different take focused more on that kind of user, eg, connected to their operational databases like Splunk, elastic, neo4j/Neptune, & databricks, and want the choice to use directly or embedded in their existing tools & automations, we are getting ready for more cohorts on louie.ai cloud, and would welcome folks curious for their team. Early enterprise & gov teams on self-hosted louie.ai face the same thing every day.. it really changes how I feel about programming.
While a lot more to do, half a year of working with LLM-augmented querying & coding already feels like a weight has been lifted from my shoulders that's been there for decades yet I never realized things can be different. Ex: "Try that last cell again, except this time trim the IP columns to the first 2 octals, and then visualize how servers connect together in a GPU graph network viz." I can in theory code the sql/spl, pandas, and python viz for that... and figure out the precise syntax/lib/code tweaks to get there... Or just write that high-level edit as if I was pairing with any reasonable person!
For a slightly different take focused more on that kind of user, eg, connected to their operational databases like Splunk, elastic, neo4j/Neptune, & databricks, and want the choice to use directly or embedded in their existing tools & automations, we are getting ready for more cohorts on louie.ai cloud, and would welcome folks curious for their team. Early enterprise & gov teams on self-hosted louie.ai face the same thing every day.. it really changes how I feel about programming.
While a lot more to do, half a year of working with LLM-augmented querying & coding already feels like a weight has been lifted from my shoulders that's been there for decades yet I never realized things can be different. Ex: "Try that last cell again, except this time trim the IP columns to the first 2 octals, and then visualize how servers connect together in a GPU graph network viz." I can in theory code the sql/spl, pandas, and python viz for that... and figure out the precise syntax/lib/code tweaks to get there... Or just write that high-level edit as if I was pairing with any reasonable person!
This looks fantastic. Is this something we can try for ourselves, or it relies on a private library?
its public observablehq notebook under MIT license. You can fork it and off you go! You have to add your OPEN_API_KEY, I am not paying for your fun, lol! But yeah, the code is all public. The burger menu on the left is where you can open/close cells for editing.
Thanks! Honestly, this is unbelievably cool. It has that rare quality of making you go: “oh, I must immediately adopt this obviously superior paradigm”. Bravo!
As its quite a steep learning curve I will add announce better instructions and UX upgrades here https://www.reddit.com/r/robocoop/