Hate Chatbots? You Aren't the Only One(wsj.com)
wsj.com
Hate Chatbots? You Aren't the Only One
https://www.wsj.com/tech/ai/chatbots-consumer-sentiment-f045b6cd
93 comments
The issue I have with most chatbots, or voice assistants on helplines, is that they're designed to be a time waster before I receive actual help. Either A) I know what I want, and that I can't get it without human intervention, or B) I can't formulate my question, and chatbots do a terrible job to point the relevant doc to me.
They seem optimized for situation C) where I need something that is easily understood/available, but am actually lazy to look it up, which doesn't really happen.
And so, my typical chatbot interaction is me trying to speedrun through the bullshit in order to get to actual help.
There was one notable exception, where the chatbot was actually there to ask 2-3 questions/give informations to prepare for the following human interaction, and made it in such a way that user saw the goal. I wish all chatbots were like that.
They seem optimized for situation C) where I need something that is easily understood/available, but am actually lazy to look it up, which doesn't really happen.
And so, my typical chatbot interaction is me trying to speedrun through the bullshit in order to get to actual help.
There was one notable exception, where the chatbot was actually there to ask 2-3 questions/give informations to prepare for the following human interaction, and made it in such a way that user saw the goal. I wish all chatbots were like that.
While it may not happen for you, “too lazy to look it up” is the vast majority of CS requests.
My understanding from talking to a couple of CS execs is that these have been a slam dunk in terms of ROI because CS agents don’t need to handle type C requests. I expect we’ll only see more as time goes on.
My understanding from talking to a couple of CS execs is that these have been a slam dunk in terms of ROI because CS agents don’t need to handle type C requests. I expect we’ll only see more as time goes on.
My guess is the ROI is provided by people giving up before they actually get help from a human.
I've analyzed support ticket requests before, and that doesn't seem to be the case. At least for the two times I've done this: 1) IT support tickets for a local school, and 2) Tickets for a B2B SaaS app. In both cases the majority of tickets where for things that seemed to me to be obvious. That if the user just bothered to spend 10 seconds looking they would figure it out. But they didn't. Some training helped on the IT side, and some UX improvements helped in SaaS app, but the bar is _sooo_ much lower than many expect.
This should be a lot more obvious to the tech crowd than it is. I suppose it's the familiarity effect (see https://xkcd.com/2501/)--what's obvious to us isn't necessarily obvious to most people, and we heavily undercount the degree to which confusion-of-basic-things exist because it's second nature to us.
And we are talking even more basic than most people on HN can imagine.
Such as:
"Is your device turned on?"
"Are you logged into the site and not just searching google for the thing your want our application to do?"
"Have you actually purchased our product and not a competitor's you just think is similar?"
Such as:
"Is your device turned on?"
"Are you logged into the site and not just searching google for the thing your want our application to do?"
"Have you actually purchased our product and not a competitor's you just think is similar?"
I wonder that too. If you're only measure one part of the funnel (e.g. CS costs) and not the total funnel (e.g. losses due to poor CS quality like a customer dropping the project) then it's easy to conclude that making CS more painful is a win.
It depends on the business, but the kind of metrics you are talking about are measured and taken seriously. People have absolutely gotten fired for CS quality KPI drops.
I don't doubt you, but if that's the case why not make it easy to get to a human? I'm fine explaining my problem to a robot, but if (when) they don't understand what I'm saying, hand me off to a human! For example, it's maddening to call the pharmacy and go through something like this:
Me: Jesus Christ, do I have to hang up and go through this whole thing ag... <cut off by robot>
Pharmacy Robot: Sorry, I didn't get that number. Using...<cut off by human hanging up>
That's just the most recent one I had. There are often better examples of madness...
Pharmacy Robot: Hello, thanks for calling <pharmacy>. What can I do for you? You can say anything like, "Check pharmacy hours" or "order a refill".
Me: Hi, I have a refill for <specific medication with rules around it> that is due next week but I'll be traveling out of the country to <other country> for a couple of weeks. I need to know what my options are.
Pharmacy Robot: Ok, you want a refill. Please enter the prescription number now.
Me: No, if we try to refill it, the automated system will just reject it. I need to talk to a h...<cut off by robot>
Me: Jesus Christ, do I have to hang up and go through this whole thing ag... <cut off by robot>
Pharmacy Robot: Sorry, I didn't get that number. Using...<cut off by human hanging up>
That's just the most recent one I had. There are often better examples of madness...
Because unless the chatbot is both better than a human in every way, and everyone knows that, the first thing people will do is push the button to reach the human. Why wouldn't they? They're calling in the first place because they don't want to make an effort to use the available tools to answer their question. They want a human.
> They're calling in the first place because they don't want to make an effort to use the available tools to answer their question.
That's not correct. I NEVER call without first exhausting every available source because I despise the phone system and it's inefficiencies. Most companies may think they have resources available, but they really don't. And no, just throwing up a zendesk or equivalent "knowledge base" isn't the same as providing tools and manuals/guides/etc.
That said, there is definitely a subset of people for whom calling is step 1 (before even googling). They tend to be older and/or on the tech illiterate side. But if you design and build for the worst-case scenario, you're really screwing over your more self-help customers and even driving them away.
That's not correct. I NEVER call without first exhausting every available source because I despise the phone system and it's inefficiencies. Most companies may think they have resources available, but they really don't. And no, just throwing up a zendesk or equivalent "knowledge base" isn't the same as providing tools and manuals/guides/etc.
That said, there is definitely a subset of people for whom calling is step 1 (before even googling). They tend to be older and/or on the tech illiterate side. But if you design and build for the worst-case scenario, you're really screwing over your more self-help customers and even driving them away.
To be fair, LLM-based chatbots are much better about this because you don't need to discover the magic incantation to talk to a human. It's a trade-off because that same property introduces the possibility of hallucination.
They're especially slam dunks when they don't provide you with a way to get out of the automated useless system. Looking at you Amazon
You can get out of the automated useless system. They don't make it easy.
But I once managed to get through to an actual agent with this question:
1. I want to buy a kindle version of this book [amazon link, for the paper version of the book].
2. On the page for the book, there is a link for the kindle edition: [link].
3. That link goes to a page for what appears to be an entirely different book. (Under the same name; this was an edition of the Arabian Nights.)
4. However, I have independently found this page: [link], which appears to be for the kindle version of the book I'm interested in.
5. Given that I want to buy the kindle version of the book linked up in step (1), which one should I purchase?
The agent directed me to buy the book that purported to be the book I wanted, instead of the book that Amazon believed was the book I wanted but which claimed to be something different. I would have assumed that anyway. But a couple days later I checked on the book and the "kindle version" link for the paper version had been corrected.
Unfortunately, while they did correct the issue on the one book that I took the time to point out to them, it's still rampant all over their website.
But I once managed to get through to an actual agent with this question:
1. I want to buy a kindle version of this book [amazon link, for the paper version of the book].
2. On the page for the book, there is a link for the kindle edition: [link].
3. That link goes to a page for what appears to be an entirely different book. (Under the same name; this was an edition of the Arabian Nights.)
4. However, I have independently found this page: [link], which appears to be for the kindle version of the book I'm interested in.
5. Given that I want to buy the kindle version of the book linked up in step (1), which one should I purchase?
The agent directed me to buy the book that purported to be the book I wanted, instead of the book that Amazon believed was the book I wanted but which claimed to be something different. I would have assumed that anyway. But a couple days later I checked on the book and the "kindle version" link for the paper version had been corrected.
Unfortunately, while they did correct the issue on the one book that I took the time to point out to them, it's still rampant all over their website.
shouting "speak to a fucking human" repeatedly seems to work, though i may be suffering from confirmation bias
Actually this does sometimes work because some of the systems now have sentiment analysis baked in and can tell if the user is getting pissed off. I've used this a few times to get through as well.
Be careful, your voice could be used to train the next chat bots, and they could start yelling angrily at customers... actually, if the new chat bot is genuinely helpful, a screaming conversation would be kind of cathartic.
It's dystopian and yet, somehow, soothing.
type `agent` repeatedly into the chatbot and it will let you request a callback
"Getting through chat bots to get to a human" is the new "getting through tech support to get to an engineer".
https://xkcd.com/806/
https://xkcd.com/806/
I think the path is now Chat bot -> Help Desk -> Engineer.
I just went through that recently, chat bot responded instantly to the mail with the same reply as the FAQ help, then the human responded after an hour asking for screenshots to see that showed I actually tried, then after a day an engineer fixed it.
I don't know all places but I've implemented a chat system before. We basically had the current employee's answering calls make lists of their most common questions along with logs and recordings we reviewed. The overwhelming majority of calls consisted of 8-10 questions and the rest could be answered with another set of 35-40 questions. Very few calls actually needed a person. We ended up with I think 80 questions it could answer in total.
This was before LLM stuff.
Funny anecdote. The most common question was "what is the price/how much does it cost." Which always was strange to me since if they were on the website with the chatbot the price was right there.
This was before LLM stuff.
Funny anecdote. The most common question was "what is the price/how much does it cost." Which always was strange to me since if they were on the website with the chatbot the price was right there.
People were likely angling for a discount.
Nope. The actual answer is: people don't like reading. We are here on Hacker News volunteering to read a giant comments section that consists entirely of words, that we're expected to read only after the article which itself consists of lots of words, so we're really weird self-selected people. Of course we find this baffling.
There was an article a while back that said something like most books sell less than 300 copies. I think the real numbers turned out to be higher, but still really low. Reading is not a popular activity. Watching video is popular. Talking to people is popular. These are skills that children acquire naturally without effort. Reading: difficult, slow, isolating. Given a choice between talking to someone or reading, people will go through a lot of pain for the opportunity to talk.
This was one of the hardest lessons to learn in my software career. I still suck at it. It's one of the lowest hanging fruits for me to improve as a developer. Just today I was shortening a popup message that my company's software sometimes puts on the screen [1] at the suggestion of a customer who had to deal with support requests from people who weren't reading it. Kicking myself!
Another popup this software rarely has to generate says "Install failed. The following error has been copied to your clipboard .... <stuff>". 100% of error reports, and I do mean 100% without exception for over two years now, come in the form of screenshots. Exactly zero people have ever read the second sentence and used paste. Kicking myself again! Why did I even think that would work?!
The hardest thing is developer docs. I still don't know how to strike the right balance there. We should make more videos.
[1] Well actually Sparkle.framework, but Conveyor incorporates it. It makes distributing desktop apps super easy, check out bio, it's cool.
There was an article a while back that said something like most books sell less than 300 copies. I think the real numbers turned out to be higher, but still really low. Reading is not a popular activity. Watching video is popular. Talking to people is popular. These are skills that children acquire naturally without effort. Reading: difficult, slow, isolating. Given a choice between talking to someone or reading, people will go through a lot of pain for the opportunity to talk.
This was one of the hardest lessons to learn in my software career. I still suck at it. It's one of the lowest hanging fruits for me to improve as a developer. Just today I was shortening a popup message that my company's software sometimes puts on the screen [1] at the suggestion of a customer who had to deal with support requests from people who weren't reading it. Kicking myself!
Another popup this software rarely has to generate says "Install failed. The following error has been copied to your clipboard .... <stuff>". 100% of error reports, and I do mean 100% without exception for over two years now, come in the form of screenshots. Exactly zero people have ever read the second sentence and used paste. Kicking myself again! Why did I even think that would work?!
The hardest thing is developer docs. I still don't know how to strike the right balance there. We should make more videos.
[1] Well actually Sparkle.framework, but Conveyor incorporates it. It makes distributing desktop apps super easy, check out bio, it's cool.
There's also another, similar group: people that make phone calls when traveling or being busy with something else.
Could be an electrician or something, driving the car from one customer to the next it's easy to wait in a phone queue and then ask someone for the information. Relative to sitting down and being useless for a while, thumbing away at some badly structured corporate FAQ, that can seem quite attractive. When you're on your break, you're on your break, you want to savour the cinnamon bun and coffee, and would rather not sully it with some boring web page.
Could be an electrician or something, driving the car from one customer to the next it's easy to wait in a phone queue and then ask someone for the information. Relative to sitting down and being useless for a while, thumbing away at some badly structured corporate FAQ, that can seem quite attractive. When you're on your break, you're on your break, you want to savour the cinnamon bun and coffee, and would rather not sully it with some boring web page.
Good point, yeah. Multi-tasking is a lot easier with voice interfaces.
Chatbots in mainstream businesses are built for one purpose: to reduce the number of staff that are needed to answer phone calls. At this, they are moderately successful. They are built for the 80%+ of customers whose enquiries are routine. As a sophisticated customer with a sophisticated enquiry that needs human attention, your time is collateral damage.
I also find chatbots to be useless time-wasters. So much so that if a company forces me to go through one to get support, I consider that to be the same as not providing support at all.
i strongly suspect the demographic of C to be much greater than we would like to belive.
then again, misanthropy is likely the core sales pitch with digitization
then again, misanthropy is likely the core sales pitch with digitization
Situation C happens all the time. I guarantee it 1000%, it's the vast majority of chats/calls/emails for many things.
I did phone support for pretty advanced technology things (you had to be pretty involved in IT to even have one of these) and still a decent number were fixed by "read the FAQ, do steps 1 and 2."
I did phone support for pretty advanced technology things (you had to be pretty involved in IT to even have one of these) and still a decent number were fixed by "read the FAQ, do steps 1 and 2."
> The issue I have with most chatbots, or voice assistants on helplines, is that they're designed to be a time waster before I receive actual help.
Yes, exactly.
And I HATE calling phone numbers for something I need. By the time I'm using the phone to call you, I've already exhausted every possible avenue I have, and I'm also already pretty irritated. At the least, make it so your chatbot tells the human what I've already said so I don't have to repeat myself once I get a human. Also, if the chatbot doesn't understand what I'm saying (which is nearly always), just send me to a human.
I think it's utterly insane that things have gotten so bad that now Google Pixel phones come with a system to handle that automated crap for you. We're now building bots to talk to the bots? I'm extremely grateful for the Google product, don't get me wrong, but the fact that it needs to exist is in my opinion a great example of how terrible things are with phone support.
Yes, exactly.
And I HATE calling phone numbers for something I need. By the time I'm using the phone to call you, I've already exhausted every possible avenue I have, and I'm also already pretty irritated. At the least, make it so your chatbot tells the human what I've already said so I don't have to repeat myself once I get a human. Also, if the chatbot doesn't understand what I'm saying (which is nearly always), just send me to a human.
I think it's utterly insane that things have gotten so bad that now Google Pixel phones come with a system to handle that automated crap for you. We're now building bots to talk to the bots? I'm extremely grateful for the Google product, don't get me wrong, but the fact that it needs to exist is in my opinion a great example of how terrible things are with phone support.
I've worked on chatbots for a couple of years. I too have come to the conclusion that they're generally garbage from a customer perspective.
Sometimes they can serve as a useful intake system for collecting information before passing someone off to an agent, but even then they tend to be less useful than just filling in a form. I think in most cases, a step by step form followed by a live chat would be better.
Unfortunately there is a segment of the population that is completely incapable of filling in a form. These people should probably be passed off to human beings, but then those human beings are wasting their time dealing with similar issues constantly.
LLM chatbots are even worse. They have great latitude to say convincing and incorrect information, but can rarely actually do anything. It would be much better to just publish a good knowledge base.
I would say a pet peeve of mine is when you do end up having support staff that are human beings, but are not empowered/trained to actually solve customer problems sufficiently. From a business perspective a chatbot is a great replacement for that, because it also regurgitates a script without actually having to action any business processes.
Sometimes they can serve as a useful intake system for collecting information before passing someone off to an agent, but even then they tend to be less useful than just filling in a form. I think in most cases, a step by step form followed by a live chat would be better.
Unfortunately there is a segment of the population that is completely incapable of filling in a form. These people should probably be passed off to human beings, but then those human beings are wasting their time dealing with similar issues constantly.
LLM chatbots are even worse. They have great latitude to say convincing and incorrect information, but can rarely actually do anything. It would be much better to just publish a good knowledge base.
I would say a pet peeve of mine is when you do end up having support staff that are human beings, but are not empowered/trained to actually solve customer problems sufficiently. From a business perspective a chatbot is a great replacement for that, because it also regurgitates a script without actually having to action any business processes.
> LLM chatbots are even worse. They have great latitude to say convincing and incorrect information, but can rarely actually do anything. It would be much better to just publish a good knowledge base.
The problem of course is that users have all been trained that knowledge bases are terrible and will go straight to support without even bothering to look.
Many users also don’t really want to become an expert in your thing. They just want someone to solve their problem and don’t care how
For example: I could learn the ins and outs of quickbooks, but I don’t really wanna. The once or twice per year that the UI won’t let me easily do what I want – straight to support.
The problem of course is that users have all been trained that knowledge bases are terrible and will go straight to support without even bothering to look.
Many users also don’t really want to become an expert in your thing. They just want someone to solve their problem and don’t care how
For example: I could learn the ins and outs of quickbooks, but I don’t really wanna. The once or twice per year that the UI won’t let me easily do what I want – straight to support.
>> Unfortunately there is a segment of the population that is completely incapable of filling in a form.
I work in accessibility.
Most of the forms I've worked with are horrendously bad for accessibility. Chat bots are even worse. People with vision or cognitive issues are an afterthought these days. As we've forced more and more people to use these poorly coded or poorly designed applications and moved away from allowing people to talk to an actual person, we are creating an internet that is increasingly less accessible to people who really need it.
The sad thing is way too many devs I work with have no idea how to write semantic HTML any more. Just simple things like putting things in an unordered list seems beyond some of the frameworks and CMS they use.
I work in accessibility.
Most of the forms I've worked with are horrendously bad for accessibility. Chat bots are even worse. People with vision or cognitive issues are an afterthought these days. As we've forced more and more people to use these poorly coded or poorly designed applications and moved away from allowing people to talk to an actual person, we are creating an internet that is increasingly less accessible to people who really need it.
The sad thing is way too many devs I work with have no idea how to write semantic HTML any more. Just simple things like putting things in an unordered list seems beyond some of the frameworks and CMS they use.
I agree. I think that a website using (styled) normal semantic HTML forms would be wayyy better for accessibility than some chatbot service which is likely doing some weird things to the markup as a response to stuff coming in on web sockets etc. I think Gov.UK is a great example of how those forms should look most of the time.
I would be a strong advocate for some regulatory framework requiring an effective right to access human support for businesses exceeding a certain size.
I would be a strong advocate for some regulatory framework requiring an effective right to access human support for businesses exceeding a certain size.
Stop blaming developers and start blaming your organization for not taking accessibility seriously enough.
It takes a ton of review, SQA, and dev time to get this stuff right. Writing semantic HTML is just the tip of the iceberg. Nobody is an expert on it either no matter how well they've read WCAG and how much experience they have implementing it. It's a moving target and every page or app is different especially when there are any design or functionality changes. It's a team effort. You can't just dump all responsibility on devs.
It takes a ton of review, SQA, and dev time to get this stuff right. Writing semantic HTML is just the tip of the iceberg. Nobody is an expert on it either no matter how well they've read WCAG and how much experience they have implementing it. It's a moving target and every page or app is different especially when there are any design or functionality changes. It's a team effort. You can't just dump all responsibility on devs.
> Unfortunately there is a segment of the population that is completely incapable of filling in a form. These people should probably be passed off to human beings, but then those human beings are wasting their time dealing with similar issues constantly.
> LLM chatbots are even worse. They have great latitude to say convincing and incorrect information, but can rarely actually do anything. It would be much better to just publish a good knowledge base.
Just as there is a segment of the population that is incapable of filling in a form, there is a (probably larger) segment which is incapable or at least unwilling to educate themselves with a good knowledge base. There are also a lot of bad knowledgebases with incomplete, obsolete, or misleading information out there, which doesn't help the cause.
> LLM chatbots are even worse. They have great latitude to say convincing and incorrect information, but can rarely actually do anything. It would be much better to just publish a good knowledge base.
Just as there is a segment of the population that is incapable of filling in a form, there is a (probably larger) segment which is incapable or at least unwilling to educate themselves with a good knowledge base. There are also a lot of bad knowledgebases with incomplete, obsolete, or misleading information out there, which doesn't help the cause.
> It would be much better to just publish a good knowledge base.
Yes, but a lot of companies seem to institutionally incapable of doing so. The number of FAQ pages (or worse, FAQ apps) I've come across filled with questions that no human has ever asked about a product, but instead list all the questions that the marketing department wished people would ask about their product because the answers paint it in such a good light, is astonishing.
The trouble with an actual knowledgebase that helps people who are struggling to use your product, is that it broadcasts to the world the ways in which your product is hard to use. And that just doesn't look good. So that info is either not published at all, or made deliberately hard to find, in order that people can't figure out where the warts are.
Yes, but a lot of companies seem to institutionally incapable of doing so. The number of FAQ pages (or worse, FAQ apps) I've come across filled with questions that no human has ever asked about a product, but instead list all the questions that the marketing department wished people would ask about their product because the answers paint it in such a good light, is astonishing.
The trouble with an actual knowledgebase that helps people who are struggling to use your product, is that it broadcasts to the world the ways in which your product is hard to use. And that just doesn't look good. So that info is either not published at all, or made deliberately hard to find, in order that people can't figure out where the warts are.
> Sometimes they can serve as a useful intake system for collecting information before passing someone off to an agent, but even then they tend to be less useful than just filling in a form. I think in most cases, a step by step form followed by a live chat would be better.
A phenomenal amount seem to collect useful identifying information, and then the agent ... asks you for the same information again? I don't know why.
A phenomenal amount seem to collect useful identifying information, and then the agent ... asks you for the same information again? I don't know why.
LLMs could be a great benefit to everyone by making chatbots not shit, if they were used for what they excel at - parsing unstructured natural language inputs, and mapping them to a finite set of (parameterized) options. You could skip the generative part for the bot reply entirely - having a chatbot that understands what I want would be like night and day, even if all it does later is telling me they can't help me.
> You could skip the generative part for the bot reply entirely
How could you remove the generative part from an LLM? (genuine question)
How could you remove the generative part from an LLM? (genuine question)
Sorry, bad phrasing. What I mean is, LLMs are more useful when they're asked to parse user input, into a more structured and narrow output, that is then fed to some business logic. Like, there's countless different ways the user could ask about their missing shipment; unlike earlier methods, LLMs are able to correctly classify almost all of them as "missing shipment". So do this, and then stop at that. The conversational output you send to the user does not need to be generated by an LLM. There's not much value added for the massive risk of hallucinations LLMs introduce.
I've now encountered what are obviously ChatGPT-powered chatting tools on multiple product websites.
Each time I asked the tool a question, it responded with completely unrelated garbage, in one case even spitting out fake tracking info for an order I never made. This looks SO BAD, I'm truly baffled that any company has signed on to use such a service.
Each time I asked the tool a question, it responded with completely unrelated garbage, in one case even spitting out fake tracking info for an order I never made. This looks SO BAD, I'm truly baffled that any company has signed on to use such a service.
Ask Air Canada how they feel about them.
I recently needed to reset a rachio device because it was tied to a previous owner and you can't do that yourself.
Rachio has replaced their customer service with an LLM that can only link you help pages. 100% useless. If I hadn't been able to find their real phone number from Google I wouldn't have been able to use their device.
Really made me question buying anything from them
Rachio has replaced their customer service with an LLM that can only link you help pages. 100% useless. If I hadn't been able to find their real phone number from Google I wouldn't have been able to use their device.
Really made me question buying anything from them
I wonder how could they possibly fuck that up so bad. ChatGPT is a dream tool that can easily parse natural language into a fixed number of structured queries - which is the primary function of all the ordinary chatbots, and the one they suck at. Those companies got handled a magic, drop-in solution to the technical difficulties of the bot flow, and somehow managed to use it to make the problem worse. Takes some really special skills.
It's a new tool. Give devs time. Bear in mind that accurate tool use is still a very new capability, one that many devs don't realize exists yet, and one that cheap models like GPT-3 have historically struggled with.
With time best practices and common patterns will get established.
With time best practices and common patterns will get established.
I suppose maybe the businesses have different priorities than the users? Wouldn't be the first time.
As a user, most of my interactions with pre-ChatGPT breed of chatbots (including "voice bots" on the phone) involved the robot having the capacity to help me at least partially, but failing to understand my request. Fixing that doesn't even require accurate tool use - it requires using LLM on input to parse it in place of whatever they're currently using, instead of directly as a chatbot; something even pre-GPT-3.5 completion models were somewhat good at.
I'm all for giving people time with new technologies, but here, 1) somehow they managed to take what should be an out-of-the-box improvement and use it to make things worse, and 2) myself and people like me are on the receiving end of the problem. At scale, there's lots of extra real frustration being created, and lots of additional people-years wasted, through companies jumping on such tools to potentially save a buck by automating away a few jobs. Hell, it probably shows as healthy profit on the books, as it just further disempowers customers, who learn to take a beating instead of fighting, because what's the point anymore...
As a user, most of my interactions with pre-ChatGPT breed of chatbots (including "voice bots" on the phone) involved the robot having the capacity to help me at least partially, but failing to understand my request. Fixing that doesn't even require accurate tool use - it requires using LLM on input to parse it in place of whatever they're currently using, instead of directly as a chatbot; something even pre-GPT-3.5 completion models were somewhat good at.
I'm all for giving people time with new technologies, but here, 1) somehow they managed to take what should be an out-of-the-box improvement and use it to make things worse, and 2) myself and people like me are on the receiving end of the problem. At scale, there's lots of extra real frustration being created, and lots of additional people-years wasted, through companies jumping on such tools to potentially save a buck by automating away a few jobs. Hell, it probably shows as healthy profit on the books, as it just further disempowers customers, who learn to take a beating instead of fighting, because what's the point anymore...
Yeah, I hear you. I think you see using LLMs for parsing only as intuitive and obvious, but I bet most devs don't think of doing it that way, at least not at first. After all, the thing is right there, willing to talk to you, and the most obvious way to use them is to take the words generated and send them to the customer. Using LLMS to generate tool invocations that then generate templated words is a much more indirect use, and frankly up until recently is not always practical due to the amount of context window that all the tool descriptions and possible actions can take up (which must fit not only the 'training' for the model but also the customer's conversation so far).
As tool use becomes more widespread and context window sizes go up, I bet we'll start to see blog posts where devs talk about how they made your approach work well, and then others will see it makes sense and start to copy it.
As tool use becomes more widespread and context window sizes go up, I bet we'll start to see blog posts where devs talk about how they made your approach work well, and then others will see it makes sense and start to copy it.
Funny. I sit next to someone at work on a distant team who spends all day in meetings talking about how chatbots for tech support are preferred over the real thing
It's like some people have lodged themselves into this niche and want to bullshit some kind of demand for it into existence
It's like some people have lodged themselves into this niche and want to bullshit some kind of demand for it into existence
Context always matters. I have no trouble imagining scenarios in which I would prefer a chatbot, and scenarios where I would prefer or even require a human.
However, one way in which chatbots are often preferred: They cost less to the business. Whether they're good or not seems to be a secondary concern. So, if your coworker is describing what the people inside the business want, they're probably right!
However, one way in which chatbots are often preferred: They cost less to the business. Whether they're good or not seems to be a secondary concern. So, if your coworker is describing what the people inside the business want, they're probably right!
> I have no trouble imagining scenarios in which I would prefer a chatbot
I'm baffled, which scenarios?
I'm baffled, which scenarios?
Any shallow decision tree where the nuance doesn't matter.
An example might be:
There's something wrong with my order -> wrong size -> order not shipped -> update size on order to small -> small out of stock -> cancel or backorder
or
There's something wrong with my order -> wrong size -> sorry, the order already shipped -> start a return
You can argue that these kinds of things should be supported via direct web UI, and that's fine, but I've seen them backed by chatbots, and I would rather do it on the web than make a phone call for a boring decision tree.
An example might be:
There's something wrong with my order -> wrong size -> order not shipped -> update size on order to small -> small out of stock -> cancel or backorder
or
There's something wrong with my order -> wrong size -> sorry, the order already shipped -> start a return
You can argue that these kinds of things should be supported via direct web UI, and that's fine, but I've seen them backed by chatbots, and I would rather do it on the web than make a phone call for a boring decision tree.
there are a lot of non-native speakers employed in the callcenter business.
Let me add a little context: the use case is mostly support when people are facing infra issues
And "preferrable" meant it was preferrable to developers specifically, which based off of my experience nobody wants it there, it's an annoyance at best
Whether it saves cost or not: hiring people to manage the automated thing (including aforementioned bullshitter who is most likely expensive) or cheap front line support, I'm not sure
And "preferrable" meant it was preferrable to developers specifically, which based off of my experience nobody wants it there, it's an annoyance at best
Whether it saves cost or not: hiring people to manage the automated thing (including aforementioned bullshitter who is most likely expensive) or cheap front line support, I'm not sure
>> want to bullshit some kind of demand for it into existence
Some blockchain people have leveraged that into lucrative careers, albethey short.
Some blockchain people have leveraged that into lucrative careers, albethey short.
Technical support with Comcast yesterday: https://imgur.com/a/NJMEIkm
Their actual human support couldn't figure out my problem, but instead of admitting it they just turn on AI-chatbot mode, effectively hanging up on the customer. There's no UI indication you're not talking to a human anymore. Meanwhile the AI tries to engage in small talk and tells you to patiently wait while they're fixing your problem. Eventually an actual human came back and admitted they couldn't fix the problem and was escalating me to "tier-2 support", which turned out to be another chatbot. Incredibly dystopian experience.
Meanwhile their engineers probably got a promotion for this and wrote an arxiv post bragging about their work: https://arxiv.org/html/2405.00801v2
Their actual human support couldn't figure out my problem, but instead of admitting it they just turn on AI-chatbot mode, effectively hanging up on the customer. There's no UI indication you're not talking to a human anymore. Meanwhile the AI tries to engage in small talk and tells you to patiently wait while they're fixing your problem. Eventually an actual human came back and admitted they couldn't fix the problem and was escalating me to "tier-2 support", which turned out to be another chatbot. Incredibly dystopian experience.
Meanwhile their engineers probably got a promotion for this and wrote an arxiv post bragging about their work: https://arxiv.org/html/2405.00801v2
Heavenbanning in practice.
The AI hype means we have record number of companies confusing optimizing for their needs with optimizing for the customer. I have worked at one place that did this, and slowly but surely, everything they did to improve "scalability" chased the customers away.
If you make me interact unnecessarily with a chat bot, I dislike your company more, instantly. If a competitor doesn't and you are largely interchangeable, I'm leaving. The fact that Gen AI cuts down your support costs by making support worse is a net negative to me, the customer.
If you make me interact unnecessarily with a chat bot, I dislike your company more, instantly. If a competitor doesn't and you are largely interchangeable, I'm leaving. The fact that Gen AI cuts down your support costs by making support worse is a net negative to me, the customer.
I'm absolutely hating the chatbot item search thing on Amazon... the answers are pretty useless to say the least. I swear google and Amazon are both getting significantly worse with time/age.
We went from having pre-answering services that you had to wade through huge decision trees to get to a real person. Now people seem to think chat bots are SO MUCH better?
All we did was replace one horrible technology for another technology that does the same thing. Its not better, its not getting better and layering LLM's and AI on top of an already bad technology isn't the answer.
All we did was replace one horrible technology for another technology that does the same thing. Its not better, its not getting better and layering LLM's and AI on top of an already bad technology isn't the answer.
So people Love chatGPT , but they hate ... bots?
The only difference between chatGPT and your support bot, is lack of maturity/investments to bring their unique knowledge to have similar quality of answers as chatGPT can answer you about your queries.
The quality of the bot = money you spend * how god is the tech.
prev gen bots are still essentially a programs written by human. Build a good bot is hard , sometimes require entire team of 10+ people working every day.
Think of it on a scale :
cheap no name brand - no support
average brand - some support, almost 100% bots and documentation, super hard to get to human agent.
apple brand - ok support with bots
luxury brands - super knowledgable human been always ready to help
It's clear that we don't have resources to deliver everyone support of a luxury brand.
Luckily, we have LLM's will 100x improvements to deliver bots that are way way better.
That will allow even small brands to have a very decent support.
Chat interfaces will eventually eat western world Web 2.0, and the rest of the world is already in chat interfaces around the globe anyways.
The only difference between chatGPT and your support bot, is lack of maturity/investments to bring their unique knowledge to have similar quality of answers as chatGPT can answer you about your queries.
The quality of the bot = money you spend * how god is the tech.
prev gen bots are still essentially a programs written by human. Build a good bot is hard , sometimes require entire team of 10+ people working every day.
Think of it on a scale :
cheap no name brand - no support
average brand - some support, almost 100% bots and documentation, super hard to get to human agent.
apple brand - ok support with bots
luxury brands - super knowledgable human been always ready to help
It's clear that we don't have resources to deliver everyone support of a luxury brand.
Luckily, we have LLM's will 100x improvements to deliver bots that are way way better.
That will allow even small brands to have a very decent support.
Chat interfaces will eventually eat western world Web 2.0, and the rest of the world is already in chat interfaces around the globe anyways.
What we all had 30y ago is now a luxury. Good. Keep going...
How can people possibly be surprised by this? It’s just another layer of obstacles between you and someone might be able to help and take action.
At the absolute best, it could be an alternative to sifting through poorly written help docs.
At the absolute best, it could be an alternative to sifting through poorly written help docs.
I love chatbots, especially the ones I create and integrate with whatsapp.
They provide a great user interface, probably the best for creating and maintaining workflows. I still believe the prime time for chatbots is yet to come.
They provide a great user interface, probably the best for creating and maintaining workflows. I still believe the prime time for chatbots is yet to come.
Can you give some examples?
Sure. I created a few CLI tasks that I use frequently. One of them creates and sub videos for me to post on instagram with optimized content from LLMs like captions, hashtag, description.
Those videos are created using videos and images I get from whatsapp. So to create a new video I forward those media files to my bot which is running in my computer at home and it generates the video, description, hastags, caption and send to myself (I use 2 different numbers, one for me and one for the bot). This saves me many hours of work.
This same chatbot also helps me manage my whatsapp groups.
Those videos are created using videos and images I get from whatsapp. So to create a new video I forward those media files to my bot which is running in my computer at home and it generates the video, description, hastags, caption and send to myself (I use 2 different numbers, one for me and one for the bot). This saves me many hours of work.
This same chatbot also helps me manage my whatsapp groups.
Ahh I see. The time saving part of having it as a chatbot is the way you can easily forward things to it for it to execute a workflow.
I thought you meant more like a way of constructing or managing workflows.
I thought you meant more like a way of constructing or managing workflows.
This article could just as well be from 2016[0], when Facebook first launched chatbots for their Messenger. I played around with NLP tools like Dialogflow back then to build one of these.
We've come a long way since, but some (UX and tech) challenges remain the same.
[0] https://techcrunch.com/2016/05/29/why-do-chatbots-suck/?gucc...
We've come a long way since, but some (UX and tech) challenges remain the same.
[0] https://techcrunch.com/2016/05/29/why-do-chatbots-suck/?gucc...
Weird argumentation.
The ultimate differentiator is that LLM, with proper management, can give you a definite, exact answer to your very specific question. This is also a usability boost; it's like moving from a landline phone to a mobile phone from a usability standpoint.
The entire chatbot history is a path towards making support better and more accessible because human agents cost a lot. In fact, the entire chatbot business has to compete with the cheapest human agents on the planet.
You can clearly see NLP-gen bots evolution. From a basic how to reduce the load on human agents to cover the top 100+ questions by bot.
The current LLM generation of bots will evolve into pre-AGI bots, which will be very informative and capable of answering 1000+ questions.
The improvement is 100x, at least. This is HUGE. The numbers are just average, but the limiting factor were levels of efforts to manage these bots.
People who wrote and read the article have a professional deformation/bias.
We are all advanced Google search users or even represent businesses that depend on Google. The majority of the population on earth is still not very good at doing online research to just solve their basic problems. They trust phone calls, tiktok recommendations and sometimes chatbots, sure. They will prefer anything but not complex googling and digesting large amounts of information.
LLM-powered bots are also much easier to manage, including capabilities of bots actually doing something, e.g., making requests to DB, etc.
Even pre-AGI chatbots, with their 100x+ improvement, will cut a huge amount of what we use to call web2.0/3.0.
LLM is just a vehicle that eventually standardizes content mechanisms. It will cover every aspect of content creation,update, moderation, analytics, and delivery, including, but not only delivery through a chatbot.
Such a change will be as different as the pre-internet media vs the social media era.
We used to think that the Western world is a cutting edge of every single trend globally. In fact only western world is still living in pre-chat world where websites are the main engine of information exchange.
Look at Wechat, Telegram, and Line with their billions of auditory. LLM-powered chatbots will bring the lagging Western world, which used to be golden billion, into modern world reality. This article is just a mental resistance to what's imminent.
The ultimate differentiator is that LLM, with proper management, can give you a definite, exact answer to your very specific question. This is also a usability boost; it's like moving from a landline phone to a mobile phone from a usability standpoint.
The entire chatbot history is a path towards making support better and more accessible because human agents cost a lot. In fact, the entire chatbot business has to compete with the cheapest human agents on the planet.
You can clearly see NLP-gen bots evolution. From a basic how to reduce the load on human agents to cover the top 100+ questions by bot.
The current LLM generation of bots will evolve into pre-AGI bots, which will be very informative and capable of answering 1000+ questions.
The improvement is 100x, at least. This is HUGE. The numbers are just average, but the limiting factor were levels of efforts to manage these bots.
People who wrote and read the article have a professional deformation/bias.
We are all advanced Google search users or even represent businesses that depend on Google. The majority of the population on earth is still not very good at doing online research to just solve their basic problems. They trust phone calls, tiktok recommendations and sometimes chatbots, sure. They will prefer anything but not complex googling and digesting large amounts of information.
LLM-powered bots are also much easier to manage, including capabilities of bots actually doing something, e.g., making requests to DB, etc.
Even pre-AGI chatbots, with their 100x+ improvement, will cut a huge amount of what we use to call web2.0/3.0.
LLM is just a vehicle that eventually standardizes content mechanisms. It will cover every aspect of content creation,update, moderation, analytics, and delivery, including, but not only delivery through a chatbot.
Such a change will be as different as the pre-internet media vs the social media era.
We used to think that the Western world is a cutting edge of every single trend globally. In fact only western world is still living in pre-chat world where websites are the main engine of information exchange.
Look at Wechat, Telegram, and Line with their billions of auditory. LLM-powered chatbots will bring the lagging Western world, which used to be golden billion, into modern world reality. This article is just a mental resistance to what's imminent.
Nice sentiment, but are you using chatbots now and having a positive experience with them?
Some of them, sure. Some of them are very bad.
But it's not the bots are bad.
The bot experience = $_investment * tech efficiency.
I've been in the chatbot industry for the last 6 years.
If you disable event past-gen NLP bots, millions of people will never be able to talk to a human agent or get any support at all. Unless companies invests 10x-100x more in customer care.
Even companies like Apple, in a premium segment of margins, use community forums and bots to manage their level of support.
LLM bots will bring us to a different reality when you can have 100x better customer support with the same level of investments.
And btw, that's why google is also moving this way.
But it's not the bots are bad.
The bot experience = $_investment * tech efficiency.
I've been in the chatbot industry for the last 6 years.
If you disable event past-gen NLP bots, millions of people will never be able to talk to a human agent or get any support at all. Unless companies invests 10x-100x more in customer care.
Even companies like Apple, in a premium segment of margins, use community forums and bots to manage their level of support.
LLM bots will bring us to a different reality when you can have 100x better customer support with the same level of investments.
And btw, that's why google is also moving this way.
My only experience with chatbots outside of Chatgpt/Google was with comcast yesterday, in which it tried to convince me for hours that it was a human fixing my internet connection, and I just had to wait patiently. Comcast mixed together real humans with bots, so you never knew if you're just talking to a human helping you or a bot, just that you're in the seventh ring of hell for customer support.
I'm souring quickly on the concept
I'm souring quickly on the concept
SwissCom do this well. When you sign up you can get a discount if you agree that all your support requests have to go via a chatbot first. Or rephrased, you can agree to pay more to get priority access to human support. So the ISP cuts you in on their savings - fair.
I plopped for the chatbot, then I had a problem. My apartment is wired weirdly, long story. The chatbot was very clear that it was a bot, it worked well, and it only took a few minutes of chatting before it decided to escalate to human support. The human support called me, so I didn't have to pay any phone bill, and they did it within a few minutes of bot making the decision to escalate.
All in all it was pretty satisfying. Comcast's version sounds ... less satisfying. The arXiv paper is really the cherry on top.
I plopped for the chatbot, then I had a problem. My apartment is wired weirdly, long story. The chatbot was very clear that it was a bot, it worked well, and it only took a few minutes of chatting before it decided to escalate to human support. The human support called me, so I didn't have to pay any phone bill, and they did it within a few minutes of bot making the decision to escalate.
All in all it was pretty satisfying. Comcast's version sounds ... less satisfying. The arXiv paper is really the cherry on top.
Has anyone seen some top-notch chat bots recent to the restaurant space for helping people do phone orders? Some people here describe uses of chatbot that are meant to basically thwart customers and waste time, but IMO chatbots are getting good enough to eliminate some customer service roles.
Big question, what problem are we really trying to solve with customer support chatbots? Goes back to management and not the technology. I'm working on chatbots. Curious what would you like the bot to do when you contact customer support?
My biggest issue is that so many chatbots are utter trash, that when I come across one that is actually good and can solve my problem quicker than a human that I've already lost patience and don't want to engage with it.
If chatbots really were good, and could solve my CS problems 90%+ of the time then I'd love them. But that doesn't happen, so I want to speak to a human please.
If chatbots really were good, and could solve my CS problems 90%+ of the time then I'd love them. But that doesn't happen, so I want to speak to a human please.
I've had good experiences.
Sometimes they answer my question.
Other times they don't, and I have to wait in the support queue for a meatbag to answer.
Worth the risk of a wasted 25 seconds.
Sometimes they answer my question.
Other times they don't, and I have to wait in the support queue for a meatbag to answer.
Worth the risk of a wasted 25 seconds.
The main problem is that everybody understands that the chat bots are there to add more obstacles between you and an actual human who might be able to help. The reasons are also also easy to understand and everyone knows them. Humans cost money. Predictably, people are infuriated when obstacles are between them and what they need done.
In an ideal world, any company that sells anything would be required to have a phone number that is answered by a human. The maximum hold time would be limited by law and be required to be lower, the higher the cost of the items they sell. The punishment for violation would be jail time for the CEO, at a month per violation or so.
In an ideal world, any company that sells anything would be required to have a phone number that is answered by a human. The maximum hold time would be limited by law and be required to be lower, the higher the cost of the items they sell. The punishment for violation would be jail time for the CEO, at a month per violation or so.
> Humans cost money. Predictably, people are infuriated when obstacles are between them and what they need done.
Another way to frame this: it's a fight over whose time might be wasted.
Another way to frame this: it's a fight over whose time might be wasted.
Well, one of the group is paid for that time. The other one is paying with their time, on top of what they already paid in money.
With time being the one finite resource for everyone, stalling someone to make them go away feels unethical, when the alternative would be to just tell them to GTFO.
With time being the one finite resource for everyone, stalling someone to make them go away feels unethical, when the alternative would be to just tell them to GTFO.
[deleted]
>The main problem is that everybody understands that the chat bots are there to add more obstacles between you and an actual human who might be able to help.
Technically competent customers who have exhausted all means of self-solving the problem understand this. People that call or open a customer support chat for say, a routine return to Amazon instead of clicking a button might get upset at it but certainly don't realize they're why the bots exist.
>The punishment for violation would be jail time for the CEO.
Comments on this site read more like Reddit every day. :(
Technically competent customers who have exhausted all means of self-solving the problem understand this. People that call or open a customer support chat for say, a routine return to Amazon instead of clicking a button might get upset at it but certainly don't realize they're why the bots exist.
>The punishment for violation would be jail time for the CEO.
Comments on this site read more like Reddit every day. :(
Honestly, this has massively changed with the introduction of ChatGPT. I've used an intercom (Fin) based support function on two separate occasions and they both got me the answer (and it was not in the knowledge base, it was clearly based on a previous support chat someone else had).
I'm honestly at the point where I wish there were a built-in one-click filter for Hacker News to prevent anything related to Machine Learning, AI, Chatbots, etc, from showing up in the feeds here when I'm signed in.
[deleted]
[deleted]