HackerTrans
TopNewTrendsCommentsPastAskShowJobs

sumanyusharma

no profile record

Submissions

Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

129 points·by sumanyusharma·2 lata temu·66 comments

New code-focused LLM needle in the haystack benchmark

github.com
6 points·by sumanyusharma·2 lata temu·1 comments

Can LLMs find bugs in large Python codebases?

hamming.ai
5 points·by sumanyusharma·2 lata temu·3 comments

Show HN: Open sourcing Restly – GPT-4 powered tutorials from your OpenAPI spec

github.com
6 points·by sumanyusharma·3 lata temu·0 comments

comments

sumanyusharma
·w zeszłym roku·discuss
Congratulations on the launch. Few qs:

How do your agents decide a suspected issue is a validated vulnerability, and what measured false-positive/false-negative rates can you share?

How is customer code and data isolated and encrypted throughout reconnaissance, exploitation, and patch generation (e.g., single-tenant VPC, data-retention policy)?

Do the agents ever apply patches automatically, or is human review required—and how does the workflow integrate with CI/CD to prevent regressions?

Ty!
sumanyusharma
·2 lata temu·discuss
How is this different from Integuru? They posted a few weeks back here: https://news.ycombinator.com/item?id=41983409
sumanyusharma
·2 lata temu·discuss
Appreciate the info; I'll double-check Fidelity again!
sumanyusharma
·2 lata temu·discuss
I'm actually pretty interested in what you're building. Sure, Vanguard and Fidelity are well-established giants, but they've barely moved beyond standard ETFs for decades. Having the option to tweak weightings at a more granular level and do daily tax-loss harvesting at scale seems like a genuine step forward.

I also like that you're transparent about how you might eventually introduce additional revenue streams like margin lending or maybe even PFOF. Knowing that upfront is better than a sudden terms-of-service surprise down the road. Still, I'd hope you'll consider giving users some say over how their shares are handled — like opting out of lending — so your incentives stay aligned over the long run.

Congrats on hitting $10M AUM. I'm rooting for more low-fee alternatives that keep the user in the loop!
sumanyusharma
·2 lata temu·discuss
No plans for acquisition :)

Building product, talking to customers and making something people want!
sumanyusharma
·2 lata temu·discuss
We're focused on end-to-end evals focused on function-call accuracy, style, tone & latency of the conversations between our sims and your voice agent. Less focused on pure TTS evals at the moment!
sumanyusharma
·2 lata temu·discuss
Pipecat looks awesome! I'll run the examples over the weekend and try to see what the integration hooks need to look like: https://github.com/pipecat-ai/pipecat/tree/main/examples

It should be pretty straightforward at first glance!
sumanyusharma
·2 lata temu·discuss
Likely outsourced call centers since call complexity is low to medium. We also expect rapid adoption in industries like customer service, healthcare, and retail, where 24/7 availability could be high-impact for businesses and convenient for consumers!
sumanyusharma
·2 lata temu·discuss
Should be fixed now; could you try again please?
sumanyusharma
·2 lata temu·discuss
We forgot to enable non-US numbers in our config for the demo. (oops)

We're working on a fix right now!
sumanyusharma
·2 lata temu·discuss
I am curious - how was the team solving this at Kea?
sumanyusharma
·2 lata temu·discuss
I use Superwhisper (no affiliation, just a happy user), which runs a local Whisper model, to create most of my email drafts and post-meeting notes. I find Whisper more accurate than Mac’s built-in speech-to-text, plus I’m faster at speaking than typing.

Sometimes, I even ‘talk’ into Cursor’s chat window instead of typing. The only downside? It can get a bit annoying for others when you're talking to yourself all day.
sumanyusharma
·2 lata temu·discuss
I'm curious to learn more about what's blocking the widespread adoption of the LLM capabilities. Lack of knowledge, reliability, or something else?
sumanyusharma
·2 lata temu·discuss
This tracks. Text evals to test core logic and voice evals for overall end-to-end performance!
sumanyusharma
·2 lata temu·discuss
It's a bit of a catch-22.

Making current voice agents reliable is incredibly time-consuming and complex. This challenge has kept many teams from pushing their agents into production. Those who do launch often release a very limited, basic version to minimize risk. We frequently talk to teams in both camps.

As a result, there aren't many 'killer' voice products on the market right now. But as models improve, we'll see more voice-centric companies emerge.

Teams are already calling their agents by hand and keeping track of experiment runs in a spreadsheet. We're just automating the workflow and making it easier to run experiments!
sumanyusharma
·2 lata temu·discuss
Our customers, who build voice agents, are often asked by their customers to make their voice agents more human-like and flexible. Their clients — businesses like pest control and automotive repairs — value providing a personalized experience but want the convenience and reliability of a 24/7 booking and answering service.
sumanyusharma
·2 lata temu·discuss
Bolna looks awesome! We've considered going open-source, but we're not sure how to effectively manage a community.

I'll reach out async!
sumanyusharma
·2 lata temu·discuss
Absolutely agree that creating effective evals requires domain expertise. Right now, we're co-building evals with customers, but we're identifying which aspects can be productized.

Regarding text-based evals — part of testing voice agents involves assessing their core reasoning logic. To do that, we bypass the voice layer and simulate conversations via text. So yes, the core simulation engine is reusable for both conversational text and voice interactions.

We're also excited about shipping the ability to replay a simulated conversation inspired by a real user!
sumanyusharma
·2 lata temu·discuss
Yes! We're aiming to build a tool that both engineers and non-engineers love.

We've discovered that it's often faster for non-technical domain experts to iterate on prompts in a structured, eval-driven way, rather than relying on engineers to translate business requirements into prompts.

While storing prompts in code offers version control benefits, it can hinder collaboration. On the other hand, using a pure CMS for prompts enhances collaboration but sacrifices some modern software development practices.

We're working towards a solution that bridges this gap, combining the best of both approaches. We're not there yet, but we have a clear roadmap to achieve this vision!
sumanyusharma
·2 lata temu·discuss
I wonder if a more optimistic version of this could be used to train humans and improve their skills. I'm thinking along the lines of LeetCode / Project Euler, but more dynamic and personalized!

Few examples:

1) Customer service: Simulating challenging customer interactions could help reps develop patience and problem-solving skills.

2) Emergency responders: Creating realistic crisis scenarios (like 911 calls) that could improve decision-making under pressure.

3) Healthcare: Virtual patients with complex symptoms could speed up the learning rate for med students.

4) Conflict resolution: Practicing with difficult personalities could aid mediators and negotiators.

5) Sales: AI-simulated tough customers could help salespeople refine their pitches and objection-handling skills in a low-stakes environment.

Thoughts?