How we think about it? It asks clarifying questions before generating anything. It explains every step, what it’s doing and why. It keeps humans in control of execution.
Counterintuitive truth is that users trust it more when it slows down.
Now the hard part, how do you design friction that feels like safety, not bureaucracy?
Still figuring it out. Anyone else wrestling with this?
While analyzing data you tweak logic, redefine metrics, and try again.
Yet we often run this entire process directly on full datasets.
You usually don't need all the data
Early on, you’re testing logic, not scale and a small, truly random sample is enough for that stage.
•Queries run instantly
•You can iterate without worrying about cost
•You're more willing to experiment
A sandbox is not "dev but smaller"
A real sandbox should be cheap, fast, and easy to throw away. It's the place where you:
•Try new ideas
•Break things on purpose
•Ask "what if?" without consequences
Most teams skip this step and jump straight from idea to production logic. That's why analytics work often feels heavier than it needs to be.
At Yorph, we treat small, random sandbox datasets as a default step before scale testing, not to replace full-dataset validation, but to get the logic right first. Analytics work needs space to explore, and sandboxes make that possible.
The hard part about analyzing data isn’t generating a a series of analysis steps, it’s proving it’s correct.
Our current approach:
- Sandboxed / sample runs on smaller datasets before full execution
- Step-level transparency: summaries, intermediate tables, and generated code are all visible
- Parallel and sequential test-time execution to surface inconsistencies
- dbt-style pipelines for reproducibility and explicit dependencies
- Decomposing analyses into small, verifiable steps to avoid error compounding (similar to MAKER-style approaches)
- Online validation checks on intermediate and final outputs that trigger re-analysis when assumptions are violated
- A gradually evolving semantic layer to improve consistency and governance over time
Curious how others think about this: what would make you trust an AI-driven data platform?
Good question - Currently our approach to semantic layer creation is based on source data, interaction with the agent, and publishing workflows. We recompute the semantic layer every time a user takes an action that fits in those categories. We also plan to give the user the ability to be able to edit the semantic layer. Wrt to synced or uploaded source data, we plan to alert the users, when a change is detected. Additionally, we have been contemplating version controlled semantic layers to be able to provide the user the control as well, which would help in the scenario you have mentioned
We are getting ready to launch our agentic data platform and wanted to share what we think are the most important things we've learned. Turns out that building a reliable agentic system is largely about good engineering fundamentals and clear written communication.
Here are a few lessons that stuck:
Domain knowledge is your differentiator. Whether it's tools, evals, or fine-tuning, your agent's domain knowledge is what sets you apart from being just a wrapper around an LLM. We recommend building good simulators of the environment your agent will live in to scale these capabilities.
Architecture matters. The difference between a flashy demo and a reliable product comes down to how agents are structured, their tools, callbacks, and most importantly: context management. That includes cross-agent instructions, memory, examples. Imagine giving instructions to an intern. You want them to be complete but not overwhelming.
Balance deterministic code and LLM "magic". A good production system finds the middle ground between letting the LLM cook and making sure it doesn't burn down the kitchen. This can take a lot of trial and error to find the right balance.
Use frameworks, don't rebuild them. While it can be a great learning experience to implement your own LLM-call-and-response-parsing while loop from scratch, the frameworks around today can really save you a ton of time and irritation. Stand on the shoulders of fast-evolving Agent frameworks like Google's ADK, and just fork them when you inevitably need them to do something bespoke for your special agent.
It's been a ride getting this ready for production.
If you're exploring agentic workflows for data integration, data workflow automation, and analysis, check out what we're building at Yorph AI. We've also got a short demo here showing what our product can do.
Counterintuitive truth is that users trust it more when it slows down.
Now the hard part, how do you design friction that feels like safety, not bureaucracy?
Still figuring it out. Anyone else wrestling with this?