- most people's perception of art is heavily affected by the framing (and to a lot of people ai = bad, and so they start seeing technical issues with it that could /never/ be made by Monet despite it being a Monet)
- but I think the critique here is more: even if someone recreated a Monet stroke-for-stroke, what's the value of this copy? I think the artist's personal life and context around the painting adds so much more to it compared to just being a pretty painting (perhaps this is the single most important part of what makes a painting interesting and valuable)
I used to work on post-training & evals. it's really hard to make a good eval set and catch all forms of reward hacking. Excited to see more from poolside!
An omni model seems very useful for real-time human-computer interaction, off the top of my head:
- Voice assistants
- Customer experience
- Gaming
- Meeting assistants
- Real-time coach or user assistant for using software
- Translation
- Real-time work on a computer controlled by voice (frontend / mobile dev, CAD, 3D modeling, etc)
Traditionally a lot of these use cases with LLM agents are higher latency because the model needs to wait for the speaker to finish, then decide to call a tool or respond - if they call a tool they need to process the tool result and decide if they want to call a tool or respond, etc...
software today is heavily, heavily constrained by supply. demand is basically infinite for actually good software that solves problems people have (and people always have problems).
"She rejected several applicants with PhDs and engineering backgrounds, reasoning that their level of education could not compensate for a lack of hands-on specialty coffee experience."
1. part of the moat is their guardrails and obviously they are audited and tracked. there are agents issuing refunds and more at scale right now so not sure where the skepticism comes from.. you're free to try and jailbreak them
2. another part of the value prop of these companies is figuring out how to construct the proper harness to take advantage of the lower latency of faster models while shoring up the weaker intelligence, how you blend deterministic and non-deterministic behaviors, compliance etc.
its a hard problem which is why f500 is willing to pay up
that's fair, most implementations in the industry are in the early stages and implementing a full powered agent with access to all the tools it needs is hard (very political as you can imagine). i hope over the next year you notice them getting better!
adding some context as someone who works in this space
1. most people (average, non-tech people) reach for the phone to call in for easily solvable problems. Plus, if the agent is integrated deep enough & has tools to interact with crms, you can raise the ceiling on the types of problems it can solve.
You're trying to avoid the bad customer experience of human 1 reading off their script, then they transfer you to some other department who may or may not know how to solve your problem, and the entire interaction cost the company way more than the value created, so the company is disincentivized to help customers.
2. All the companies in this space start with the outsourced BPO market for cx (multi billion market still) but the next market is going to be in revenue generation and churn prevention at scale, i.e. how do you proactively avoid customer issues, how do you upsell and generate revenue instead of reducing cost, how do you keep customers happy?
3. I think more companies will pivot to outcome based pricing on the contrary, makes it so much more measurable than seat-based and protects margins better than usage based. Plus cx is one of the few industries with very well known metrics
4. Kind of? Most companies in this space don't use native voice models which are noticeably dumber, they use transcription + a stronger text model + TTS. The majority of customers can be handled with the latest SOTA text model and you need smart context engineering to handle the long tail of more complicated asks
but even a simple impl to answer questions can knock out like 50% of callers who are tech-illiterate at 100x cheaper cost, it's just strictly better economics and better for those customers
It's always interesting seeing how HN reacts to AI CX (as someone who works in this space). Yes, the tech savvy crowd loves to say how they always ask for a human and love old school phone trees
in reality 50-80% of callers come in with easily answerable questions because they don't know how to nav the website and prefer to ask in natural language
The vast majority of callers call in to resolve their issue, and most don't care if they are speaking to a bot because they just want their issue fixed. Agents (if implemented well) are an order of magnitude more effective at resolving issues compared to a call centre worker who is reading off a script and churn within 9 months
There's also the 2nd order effs of making CX cheap. before, there is the perverse incentive of companies trying to keep you off support because each call costs them way more than the value they get. if your cost per call drops 100x you can invest in turning a cost centre into a revenue driver (+ a better experience)
Ive been preparing somewhat for this, as someone who knows they aren't a top N% engineer. My current role involves a certain amount of sales and product in addition to SWE (and luckily I find it fun to talk to customers!)
I think it's prudent for a lot of swes to think about what a future looks like where most of the job is managing and unblocking agents.
my main qualm with Ed is his analysis on the financials is decent, but he absolutely refuses to admit that the technology is useful (especially in the hands of competent users), and that all the labs are extremely compute starved due to overwhelming demand.
- most people's perception of art is heavily affected by the framing (and to a lot of people ai = bad, and so they start seeing technical issues with it that could /never/ be made by Monet despite it being a Monet)
- but I think the critique here is more: even if someone recreated a Monet stroke-for-stroke, what's the value of this copy? I think the artist's personal life and context around the painting adds so much more to it compared to just being a pretty painting (perhaps this is the single most important part of what makes a painting interesting and valuable)