We were building voice agents across multiple platforms and had no way to regression test them. Prompt tweak on one node would silently break three other conversation paths. So we built voicetest.
It imports your agent config from any supported platform into a universal IR (AgentGraph), then runs autonomous multi-turn conversation simulations against it. A simulator LLM plays the caller, your agent graph handles the routing, and an LLM judge scores transcripts against success criteria. Also supports deterministic rule tests for compliance stuff, PII leakage, required disclosures, forbidden phrases.
Write tests once and they work across platforms. Import from Retell, export to VAPI, run the same test suite. Also does format conversion between platforms if you're migrating.
Has interfaces for CLI (CI/CD), web UI, REST API, and a TUI. Results go into DuckDB so you can query them. Uses LiteLLM via DSPy so it works w/whatever provider you want
- test harness for voice agents.
- multi platform formats (Retell, VAPI, Bland, LiveKit) compile down to a unified AgentGraph IR
- import from one platform, test locally, export to another
- use litellm, DSPY to config models, if on a subscription use claudecode as a runner to avoid API call charges
- metric judges produce continuous 0-1 scores instead of binary pass/fail since a 0.65 and a 0.35 both fail a 0.7 threshold but represent very different agent behaviors.
- persist to DuckDB for querying across test history
- adding auto-healing graph mutations where failed tests propose structural + prompt changes to the agent graph and validate against a regression suite
Thank you! It came out of that feeling of I need to use this, but I don't want to have to manage it or have to learn the details more than once. Now I find myself using it daily in my workflow, not having to worry about the extra worktree folders saves enough time to make it worthwhile
Ona | Burlington, Vermont & Nairobi, onsite / remote | Senior Software Engineers, Frontend Engineers | full time | https://www.ona.io/jobs.html
Our global health platform is used on smartphones by community health workers around the world to track and monitor their patients offline and on. Our data and ML platforms analyze patient risk, coordinate humanitarian projects, and manage Covid-19 tracking and messaging campaigns. We were named one of the “50 Most Genius Companies” by TIME magazine and have partnerships with the World Health Organization, the Bill and Melinda Gates Foundation, UNICEF, the United Nations Development Program, the World Food Program, and others.
Named one of the “50 Most Genius Companies” by TIME magazine, we are a social enterprise building technology products that improve access to essential, life-saving services. We are funded by the World Health Organization, the Bill and Melinda Gates Foundation, UNICEF, the United Nations Development Program, the World Food Program, and others.
Ona's GIT fellowship is a 12-week, Nairobi-based engagement pairing experienced software engineers with a global health or humanitarian open source project.
Our Global Health mobile team stack is Android / Java / Postgres / React
Our Data Platform team stack is Django / Clojure / ClojureScript React (om -> reagent)
Our ML team stack is NiFi / Kafka / SciPy + Jupyter (prototyping) / Superset
We have full-time positions available, as well as fellowships and other opportunities to contribute, https://ona.io/jobs.html
Ona | Software Engineer | Nairobi, Kenya & New York, NY | Full Time | Onsite
We're looking for developers who want to build foundational data systems that drive change. Our team has worked on projects that record the social infrastructure of entire countries, tally the winners of national elections, reduce infant mortality, and decrease the spread of malaria. We build software that solves real problems and you will too. We were named a TIME Magazine Genius Company of 2018.
Global health stack
* Android, Java, Postgres
Data platform stack
* Python, Clojure, ClojureScript, PostGIS
Data warehouse stack
* NiFi, Kafka, Druid, Superset, React, Redux
Learn about our recent projects and what it's like to work at Ona, https://blog.ona.io/
Ona | Software Engineer | Nairobi, Kenya | Full Time | Onsite
We're looking for developers who want to build foundational data systems that drive change. Our team has worked on projects that record the social infrastructure of entire countries, tally the winners of national elections, and reduce infant mortality. We build software that solves real problems and you will too.
Global health stack
* Android, Java, Postgres
Data platform stack
* Python, Clojure, ClojureScript, PostGIS
Data warehouse stack
* NiFi, Kafka, Druid, Superset, React, Redux
Learn about our recent projects and what it's like to work at Ona, https://blog.ona.io/
Kobo is based off of onadata, https://github.com/onaio/onadata (which is actively maintained) which was in turn the continuation of formhub (which is no longer maintained).
Ona | Jakarta, Nairobi, New York, Washington DC | Multiple Positions | ONSITE | Full-time | https://ona.io/
We're hiring senior Android and Java engineers for a new Jakarta office focused on building out the OpenSRP frontline healthcare application.
We build data management platforms and mobile health applications used globally to improve child and maternal health, governance, agriculture, access to infrastructure, and government accountability. E.g. our software routed patients during the Ebola crisis in West Africa, counted votes during a Libyan election, is surveying national infrastructure in post-earthquake Nepal, and is scheduling vaccines in Bangladesh, Indonesia, Kenya, Pakistan, Zambia, and soon expanding to more countries per our collaboration with the WHO.
The stack you'll work with depends on the project, active stacks include: Clojure/Script + Om, Django + PostGIS, Android + CouchDB, React, and a Data engineering stack with NiFi + HDFS + Kafka + Druid + Superset, details here, https://blog.ona.io/general/2017/08/30/streaming-ona-data-wi...
In a technical interview we'll ask systems architecture questions, then we'll write and extend code together, then we'll describe our culture and see if it would be a good fit. We're currently hiring interaction designers and senior engineers, https://ona.io/jobs.html, [email protected]
Ona | New York, Washington DC, Nairobi | Multiple Positions | ONSITE | Full-time | https://ona.io/
We build data management platforms and tablet health applications used globally to improve child and maternal health, governance, agriculture, access to infrastructure, and government accountability. E.g. our software routed patients during the Ebola crisis in West Africa, counted votes during a Libyan election, is surveying national infrastructure in post-earthquake Nepal, and is scheduling vaccines in Bangladesh, Indonesia, Keny, Pakistan, Zambia, and soon expanding to more countries per our collaboration with the WHO.
The stack you'll work with depends on the project, active stacks include: Clojure/Script + Om, Django + PostGIS, Tomcat + CouchDB, React, and a Data engineering stack with NiFi + HDFS + Kafka + Druid + Superset, details here, https://blog.ona.io/general/2017/08/30/streaming-ona-data-wi...
In a technical interview we'll ask systems architecture questions, then we'll write and extend code together, then we'll describe our culture and see if it would be a good fit. We're currently hiring interaction designers and senior engineers, https://ona.io/jobs.html, [email protected]
Ona (https://ona.io/) | Nairobi, Kenya | Senior Software Engineer | ONSITE | Full-time
We build a data management platform and tablet health application used globally to improve child and maternal health, governance, agriculture, access to infrastructure, and government accountability. E.g. our software routed patients during the ebola crisis in West Africa, counted votes during a Libyan election, is surveying national infrastructure post-earthquake in Nepal, and scheduling vaccines in Bangladesh, Indonesia, Pakistan, Indonesia, Zambia, and other places soon.
Ona (https://ona.io/) | Nairobi, Kenya | Senior Software Engineer / Project Managers | ONSITE | Full-time
We build a data management platform and tablet health application used globally to improve child and maternal health, governance, agriculture, access to infrastructure, and government accountability. E.g. our software routed patients during the ebola crisis in West Africa, counted votes during a Libyan election, is surveying national infrastructure post-earthquake in Nepal, and scheduling vaccines in Bangladesh, Indonesia, Pakistan, Indonesia, Zambia, and other places soon.
Ona (https://ona.io/) | Nairobi, Kenya | Senior Android/Java Engineer | ONSITE | Full-time
We build a data management platform and tablet health application used globally to improve child and maternal health, governance, agriculture, access to infrastructure, and government accountability. E.g. our software routed patients during the ebola crisis in West Africa, counted votes during a Libyan election, is surveying national infrastructure post-earthquake in Nepal, and scheduling vaccines in Bangladesh, Indonesia, Pakistan, Indonesia, Zambia, and other places soon.
Ideally having worked remotely, held an intership before, or something equivalent. But I remember applying for my early dev jobs when every posting wanted 3 years experience. We don't exclude anyone by default, no hard requirements, a PR is is the best way to get our attention https://github.com/onaio.
No need for iOS at this time, but we do need Android developers. Most of our work is deployed or working with organizations and individuals in lower-income environments, so the priority is on Android. We'd like to develop equivalents of our Android apps for iOS but it's not justifiable at this stage.
Ona (https://ona.io/) | New York City | Engineering Interns, Senior Engineers | Remote OK | Full-time
We build a data management platform and tablet health application used globally to improve child and maternal health, governance, agriculture, access to infrastructure, and government accountability. E.g. our software routed patients during the ebola crisis in West Africa, counted votes during a Libyan election, and is surveying national infrastructure post-earthquake in Nepal.
It imports your agent config from any supported platform into a universal IR (AgentGraph), then runs autonomous multi-turn conversation simulations against it. A simulator LLM plays the caller, your agent graph handles the routing, and an LLM judge scores transcripts against success criteria. Also supports deterministic rule tests for compliance stuff, PII leakage, required disclosures, forbidden phrases.
Write tests once and they work across platforms. Import from Retell, export to VAPI, run the same test suite. Also does format conversion between platforms if you're migrating.
Has interfaces for CLI (CI/CD), web UI, REST API, and a TUI. Results go into DuckDB so you can query them. Uses LiteLLM via DSPy so it works w/whatever provider you want
https://github.com/voicetestdev/voicetest