I strongly disagree ;-)
The paper's line of reasoning seems to continue the endless subjective loop of assuming your viz framework has the right abstractions & defaults , which the next person will rightfully disagree with for their slightly different eval set
We found in practice:
- LLM's generate charts fine
- LLM's tweak charts fine
- LLM's take user feedback to tweak them fine
In that sense, going higher-level for abstractions, as is being argued for here, is strictly worse: it's better to give controls so the LLM can go deep and customize
In practice, we found the choice of json config language X vs json config language Y to be pretty equivalent across different charting systems (vega, plotly, perspective, etc), LLM's do them all fine
The harder parts were deciding what a good chart is (model, reasoning, context), and opposite of this approach, giving lower-level facility for doing user change requests on tweaks, interactivity, and tricky in practice, when they have a lot of data on it.
The paper's line of reasoning seems to continue the endless subjective loop of assuming your viz framework has the right abstractions & defaults , which the next person will rightfully disagree with for their slightly different eval set
We found in practice:
- LLM's generate charts fine
- LLM's tweak charts fine
- LLM's take user feedback to tweak them fine
In that sense, going higher-level for abstractions, as is being argued for here, is strictly worse: it's better to give controls so the LLM can go deep and customize
In practice, we found the choice of json config language X vs json config language Y to be pretty equivalent across different charting systems (vega, plotly, perspective, etc), LLM's do them all fine
The harder parts were deciding what a good chart is (model, reasoning, context), and opposite of this approach, giving lower-level facility for doing user change requests on tweaks, interactivity, and tricky in practice, when they have a lot of data on it.
Twitter: LMeyerov
We're 100X'ing investigations with the first GPU visual graph AI platform and now Louie.AI, the genAI-first rethink of analyst notebooks. We enjoy working with data teams all the way from from tech companies and startups to scientists and government agencies, and on problems from threat hunting, fraud, & misinformation to supply chain, user journey, and genomics. Our partners include Amazon, Nvidia, and more.
* Louie.ai: Louie connects to your data systems so analysts can use natural language to ask questions and get back data, analyses, AI models, interactive visualizations, and everything else the Graphistry platform can do
* Graph visual analytics: Gartner measured graph as a Top 5 growing data technology for the next 5 years and awarded us as the 2021 Cool Vendor in Graph
* End-to-end GPU computing: Apache Arrow & RAPIDS.ai were both voted top data projects of 2020 & 2021
* Point-and-click workflow automation: Process automation is the fastest growing enterprise industry in history
* Graph AI (GNNs) to make it easy for operational teams to take the next step of their graph journey by automating their graph insights for tasks like detection: Winner of the 2022 US Cyber Command AI alert data challenge!
... And we're hiring! LLM app/backend engineering, AI (cyber, GNNs, ...), sales engineering, platform engineering, UI, and more: https://www.graphistry.com/careers