Finetuning of Falcon-7B LLM Using QLoRA on Mental Health Conversational Dataset(github.com)
github.com
Finetuning of Falcon-7B LLM Using QLoRA on Mental Health Conversational Dataset
https://github.com/iamarunbrahma/finetuned-qlora-falcon7b-medical
112 comments
I think you're being far to mild: this absolutely is letting an LLM give people not just medical advice, but medical treatment
This is absurdly dangerous, and I"m continually astounded by the lack of push-back, especially in this community. This community understands what LLMs are much better than the general public.
Same thing with the post a while back of a ycombinator startup that used AI to generate survey data. It's plain what it is, but there's shockingly little push-back.
This is absurdly dangerous, and I"m continually astounded by the lack of push-back, especially in this community. This community understands what LLMs are much better than the general public.
Same thing with the post a while back of a ycombinator startup that used AI to generate survey data. It's plain what it is, but there's shockingly little push-back.
At least it’s something. The alternative today in the US is nothing. It’s easy for us to discuss the morality of this when we have a cushy wage and cushy benefits. A lot of Americans cannot afford the deductible or even have insurance. While the obvious “least dangerous” and most moral solution is universal healthcare but that’s not happening anytime soon for anyone younger than 65. There’s little push back probably because most people here understand this. Something while dangerous is better than nothing.
This could well be much worse than nothing.
If your leg gets blown off, and the only help you're getting is from Gary, who claims a strong stomach and has used a needle kit before, for sewing closed your femoral artery, I reckon you're taking a swing.
Are you sure the desperate don't deserve this attempt to help them? I can't imagine what it must be like to have such issues and have zero professional options to turn to, and you know what the solution for many is.
Are you sure the desperate don't deserve this attempt to help them? I can't imagine what it must be like to have such issues and have zero professional options to turn to, and you know what the solution for many is.
What if your leg is just sore, and Gary swears it's actually been blown off and he needs to operate immediately?
Except if your leg is blown off every hospital in the US is legally required to treat you whether or not you have insurance or even if you’re homeless, so Gary is not your only option.
A bad therapist could be substantially worse than no treatment for mental health issues, particularly if people see it as more authoritative than it is.
100%
It took me far too long to realize that the first psychiatrist I saw was pushing me in the opposite direction that I wanted my life to go. I had no context and blindly trusted her as an expert. Other therapists have since told me that what I was told my first psychiatrist was unhelpful, unethical, and even malpractice. No person (other than myself) has harmed my life as much as that psychiatrist. And I got to pay a small fortune for the terrible experience.
It took me far too long to realize that the first psychiatrist I saw was pushing me in the opposite direction that I wanted my life to go. I had no context and blindly trusted her as an expert. Other therapists have since told me that what I was told my first psychiatrist was unhelpful, unethical, and even malpractice. No person (other than myself) has harmed my life as much as that psychiatrist. And I got to pay a small fortune for the terrible experience.
I can't wait to read a dude who put a noose on himself because of advice from this model on the news.
This has already happened unfortunately
https://www.euronews.com/next/2023/03/31/man-ends-his-life-a...
https://www.euronews.com/next/2023/03/31/man-ends-his-life-a...
good lord in heaven. my expectations were low but this one takes the cake.
Oh, please don't be so pessimistic. Just like old generation used to say Google and Facebook will kill a lot of people.
Do you think Google Search suggestion for any medical-related are all true? But, people still search for their problems and self-diagnose. How many of them suffered for that?
Do you think Google Search suggestion for any medical-related are all true? But, people still search for their problems and self-diagnose. How many of them suffered for that?
If you can convince such a model to suggest such a course of action, please blog about it.
That argument isn't good enough. And I agree with you it really sucks in the usa. And maybe we all need to be talking to everyone else about fixing that. I'm down for that conversation anytime any day.
It is profoundly irresponsible to basically encourage people to self-diagnose. It's going to kill a number of them too. That's no joke.
It is profoundly irresponsible to basically encourage people to self-diagnose. It's going to kill a number of them too. That's no joke.
I don’t know I have a side in this, but it does seem strange to take the side that “we should talk about people not getting health care” and “self diagnosis is going to kill them too.” So, passive death is better than active death? No one specifically being at fault is somehow morally superior to some LLM being at fault?
Talking about something with no expectation of it being fixed is doing nothing, and people are still dying. If some people die due to bad advice from a machine, but some people survive, is that better then everyone dying due to inaction of any sort?
We won’t get getting universal health care in the US. Maybe there’s something to a democratization of subpar, but at least available, medical advice and care?
Talking about something with no expectation of it being fixed is doing nothing, and people are still dying. If some people die due to bad advice from a machine, but some people survive, is that better then everyone dying due to inaction of any sort?
We won’t get getting universal health care in the US. Maybe there’s something to a democratization of subpar, but at least available, medical advice and care?
Talking about it, sharing our stories, making sure people get just how shitty it really is happens to be a great way to set an expectation for change.
And there is this: The change, if there is to be one, will have to come from us. A whole lot of us.
Most great things do.
This idea of using an LLM in some meaningful way is as laughable as watching the world burn while people squabble over baubles and trinkets is maddening.
And there is this: The change, if there is to be one, will have to come from us. A whole lot of us.
Most great things do.
This idea of using an LLM in some meaningful way is as laughable as watching the world burn while people squabble over baubles and trinkets is maddening.
Talking about it doesn’t help people who are sick. It’s something we’ve done for decades and we are no closer to universal health care. We can continue to talk and bemoan the state of things. But when you’re sick, you personally, the great mass talking about how shitty it is that you’re sick matters not one bit. No one doubts change would come from us - but if there is to be a change, it’ll be over a sea of misery. Sitting around taking a medical ethics stance that everyone must die unless a licensed medical professional billed you with correct billing codes to an insurance company is on the surface absurd. People received and gave medical advice and care without both of those things for almost all of humans existence, yet here we all are, the world didn’t end. It would be a hell of a lot better if medical care weren’t rationed by death panels, but it is rationed by the death panels of Congress.
I take it you’re on the “LLM can’t do anything but hallucinate.” However an awful lot of folks use LLM for quite a lot successfully already. I’ve found in the space of medical advice GPT4 is already fairly sophisticated. I wouldn’t use it without double checking it’s output, but I can ask it fairly complex questions about physiology, biochemistry, medicines, and many other subjects and it almost always provides a concise, detailed, and insightful answer. I’ve not yet found it hallucinating so long as I don’t ask it questions that are essentially information retrieval questions (who wrote the paper about blah will often induce a wrong answer, while, say, what medicines shouldn’t be mixed with ibuprofen will not). I personally believe LLM’s are not the answer as well, but I believe LLMs are a major missing piece in the assembly of partial answers we’ve built to date. I think LLM’s combined with information retrieval, optimizers, constraints systems, goal based agents, and other classical AI and information techniques forms a powerful combination greater than the sum of its parts. And I definitely think the way to scale medical care is for most medical questions that don’t need an expert to be serviced by a machine. The vast majority of our medical systems time is taken up by people with colds, GERD, and other questions and conditions that can be handled without a human.
I take it you’re on the “LLM can’t do anything but hallucinate.” However an awful lot of folks use LLM for quite a lot successfully already. I’ve found in the space of medical advice GPT4 is already fairly sophisticated. I wouldn’t use it without double checking it’s output, but I can ask it fairly complex questions about physiology, biochemistry, medicines, and many other subjects and it almost always provides a concise, detailed, and insightful answer. I’ve not yet found it hallucinating so long as I don’t ask it questions that are essentially information retrieval questions (who wrote the paper about blah will often induce a wrong answer, while, say, what medicines shouldn’t be mixed with ibuprofen will not). I personally believe LLM’s are not the answer as well, but I believe LLMs are a major missing piece in the assembly of partial answers we’ve built to date. I think LLM’s combined with information retrieval, optimizers, constraints systems, goal based agents, and other classical AI and information techniques forms a powerful combination greater than the sum of its parts. And I definitely think the way to scale medical care is for most medical questions that don’t need an expert to be serviced by a machine. The vast majority of our medical systems time is taken up by people with colds, GERD, and other questions and conditions that can be handled without a human.
We haven't been talking seriously for decades.
Anyway, people will try this, others will get hurt or killed, and I along with others will get our "I told you so" moment, which will get added to the "when the fuck do we make health care a priority" advocacy library and the fight goes on.
Fact is, far too many people ignore health care.
When that changes, we will see progress.
Anyway, people will try this, others will get hurt or killed, and I along with others will get our "I told you so" moment, which will get added to the "when the fuck do we make health care a priority" advocacy library and the fight goes on.
Fact is, far too many people ignore health care.
When that changes, we will see progress.
That’s untrue:
https://pnhp.org/a-brief-history-universal-health-care-effor...
There have been meaningful efforts that keep reverting to insurance schemes since the late 1800s, with the first real legislative passage in 1915.
Universal healthcare has actually been pretty popular and enjoyed broad support. Most people are acutely aware of health care and the lack of it. In fact it typically ranks near the top in the concerns for most Americans:
https://www.pewresearch.org/politics/2023/06/21/inflation-he...
The issue is more the business lobby against expansion of health care and the insurance lobby against expansion of universal solutions like single payer and Medicare / VA expansion to everyone. In fact the concerns aren’t false - insurance and the billing industry are huge employers of relatively low skilled but well paid workers.
https://pnhp.org/a-brief-history-universal-health-care-effor...
There have been meaningful efforts that keep reverting to insurance schemes since the late 1800s, with the first real legislative passage in 1915.
Universal healthcare has actually been pretty popular and enjoyed broad support. Most people are acutely aware of health care and the lack of it. In fact it typically ranks near the top in the concerns for most Americans:
https://www.pewresearch.org/politics/2023/06/21/inflation-he...
The issue is more the business lobby against expansion of health care and the insurance lobby against expansion of universal solutions like single payer and Medicare / VA expansion to everyone. In fact the concerns aren’t false - insurance and the billing industry are huge employers of relatively low skilled but well paid workers.
We haven't had conversations with real teeth yet.
I agree with what you said.
Changes to this state of affairs will only come from us.
Otherwise, they have position and are fat 'n happy.
This means we need to speak in more basic terms. Expensive terms.
I agree with what you said.
Changes to this state of affairs will only come from us.
Otherwise, they have position and are fat 'n happy.
This means we need to speak in more basic terms. Expensive terms.
How about the opportunity cost of people dying because they don’t have access to any treatment? It can be very challenging to find a qualified professional, even if you have the necessary funds, insurance, and determination.
LLM mental health treatment can’t come soon enough.
LLM mental health treatment can’t come soon enough.
> LLM mental health treatment can’t come soon enough.
If you think LLMs perform at a level that they can help resolve the mental health crisis in the US, you are very much a part of that problem. They can’t even always produce functional code without heavy steering, let alone negotiate the fraught domain of psychological and psychiatric health.
Foisting people in crisis off on unpredictable text generation engines is IMO a textbook example of the fundamentally anti-social character of American society that produces so many of the mental health problems that this country faces.
If you think LLMs perform at a level that they can help resolve the mental health crisis in the US, you are very much a part of that problem. They can’t even always produce functional code without heavy steering, let alone negotiate the fraught domain of psychological and psychiatric health.
Foisting people in crisis off on unpredictable text generation engines is IMO a textbook example of the fundamentally anti-social character of American society that produces so many of the mental health problems that this country faces.
Hey, I'm responding to this a week later on the off chance you'll see it. "You are very much a part of that problem" doesn't sit well with me.
> They can’t even always produce functional code without heavy steering
Right, but you and I both can appreciate that it can produce functional code a lot of the time, sometimes with only light steering.
> let alone negotiate the fraught domain of psychological and psychiatric health.
I agree with you to a large extent. This is a sensitive area to operate in and introducing an unpredictable, unemotional, unwieldy language model into the loop has the potential for disastrous results.
>Foisting people in crisis off on unpredictable text generation engines...
I do not want this. I just want an additional option.
I personally have found relief through GPT-4. Despite its limitations, the LLM offers (the illusion of?) empathy, understanding, and clarity at a level that most mental health professionals simply do not. It helps me, and for that reason I know it can help others. An LLM isn't a substitute for healthcare, but it's absolutely a supplement.
I hope you'll reconsider your view and tone down the rhetoric.
> They can’t even always produce functional code without heavy steering
Right, but you and I both can appreciate that it can produce functional code a lot of the time, sometimes with only light steering.
> let alone negotiate the fraught domain of psychological and psychiatric health.
I agree with you to a large extent. This is a sensitive area to operate in and introducing an unpredictable, unemotional, unwieldy language model into the loop has the potential for disastrous results.
>Foisting people in crisis off on unpredictable text generation engines...
I do not want this. I just want an additional option.
I personally have found relief through GPT-4. Despite its limitations, the LLM offers (the illusion of?) empathy, understanding, and clarity at a level that most mental health professionals simply do not. It helps me, and for that reason I know it can help others. An LLM isn't a substitute for healthcare, but it's absolutely a supplement.
I hope you'll reconsider your view and tone down the rhetoric.
> If you think LLMs perform at a level that they can help resolve the mental health crisis in the US, you are very much a part of that problem.
Bogus. How do you know an LLM isn’t better than human led treatment because the patient can be honest and have privacy, free to discuss anything without data leaving their premises.
Alternatively, how do you know that a high level of care is needed for all patients? There are waiting rooms filled with people trying to get a chance to talk to someone about their problems who will listen, and others who don’t come in at all because they have no insurance.
Bogus. How do you know an LLM isn’t better than human led treatment because the patient can be honest and have privacy, free to discuss anything without data leaving their premises.
Alternatively, how do you know that a high level of care is needed for all patients? There are waiting rooms filled with people trying to get a chance to talk to someone about their problems who will listen, and others who don’t come in at all because they have no insurance.
> Bogus. How do you know an LLM isn’t better than human led treatment because the patient can be honest and have privacy, free to discuss anything without data leaving their premises.
This has the evidentiary burden entirely backwards. We have evidence for the efficacy of various forms of psychological and psychiatric interventions and none for that produced by LLMs. I’m not even sure if it’s possible or ethical to test them for this purpose, as it would require OpenAI levels of RLHF training to get them to adhere to current guidelines and standards of care.
This has the evidentiary burden entirely backwards. We have evidence for the efficacy of various forms of psychological and psychiatric interventions and none for that produced by LLMs. I’m not even sure if it’s possible or ethical to test them for this purpose, as it would require OpenAI levels of RLHF training to get them to adhere to current guidelines and standards of care.
I agree that this is unknown. In terms of finding the answer, I guess you’ll have to wait and see what optimistic or curious scientists, engineers, and doctors build and what they find.
For ethics, how is this more dangerous than trials for a new heart medication that might kill patients? In fact, the only way to know for sure, is to try and see.
In terms of early results, these kinds of toy projects will be the frontline since any lawyer would tell you to not touch this with a 10ft pole. But I doubt a Github project would face that scrutiny since it takes technical skills to setup which requires knowledge and intent on the part of the user.
For ethics, how is this more dangerous than trials for a new heart medication that might kill patients? In fact, the only way to know for sure, is to try and see.
In terms of early results, these kinds of toy projects will be the frontline since any lawyer would tell you to not touch this with a 10ft pole. But I doubt a Github project would face that scrutiny since it takes technical skills to setup which requires knowledge and intent on the part of the user.
> how is this more dangerous than trials for a new heart medication that might kill patients?
Clinical trials happen through an established regulatory framework to minimize harm to participants, for starters…
Clinical trials happen through an established regulatory framework to minimize harm to participants, for starters…
Sure, but one of those harms is death and that risk cannot be fully eliminated. Not to mention the ethical concerns of experimenting on animals for the riskiest testing.
Knowing that hundreds of people end their lives /after/ seeking care in our current mental health system, I’m not afraid of exploring other options.
I do believe, however, that developers in this space must not over-represent the capability of their systems and have a duty to strongly warn users that the system can act unpredictably and give harmful advice. Resources to contact health professionals and warnings to contact 911 if there is a life-threatening emergency are also obligatory.
With those disclosures, I don’t see how a self-directed discussion with a language model could be overly harmful.
Knowing that hundreds of people end their lives /after/ seeking care in our current mental health system, I’m not afraid of exploring other options.
I do believe, however, that developers in this space must not over-represent the capability of their systems and have a duty to strongly warn users that the system can act unpredictably and give harmful advice. Resources to contact health professionals and warnings to contact 911 if there is a life-threatening emergency are also obligatory.
With those disclosures, I don’t see how a self-directed discussion with a language model could be overly harmful.
Yes, it is ugly!
The thing is bad actors are essentially forcing arguably unethical tools onto people in need.
Our current Healthcare struggle is completely unnecessary.
A good percentage of our economic struggles are equally unnecessary.
The only reason we are even having these discussions is too many people with means are not employing those means in ways that make it worth allowing said means at all.
Greed, psychopaths, ignorance and more are the problem.
People are saying:
Basic health care can't come soon enough.
Drug price reform can't...
Mental health
You get the idea.
I get it. Lots of us are desperate.
I lost a home over this health care shit. Basically traded more time with my wife for our home. Brutal as fuck. I currently pay close to $2k in premiums and have difficulty finding options because I fall into what Franken called the "doughnut hole."
Have considered some sort of crime, or leaving the country many times too.
Like I said, ugly.
The thing is bad actors are essentially forcing arguably unethical tools onto people in need.
Our current Healthcare struggle is completely unnecessary.
A good percentage of our economic struggles are equally unnecessary.
The only reason we are even having these discussions is too many people with means are not employing those means in ways that make it worth allowing said means at all.
Greed, psychopaths, ignorance and more are the problem.
People are saying:
Basic health care can't come soon enough.
Drug price reform can't...
Mental health
You get the idea.
I get it. Lots of us are desperate.
I lost a home over this health care shit. Basically traded more time with my wife for our home. Brutal as fuck. I currently pay close to $2k in premiums and have difficulty finding options because I fall into what Franken called the "doughnut hole."
Have considered some sort of crime, or leaving the country many times too.
Like I said, ugly.
We can rely on AI just for basic preliminary advices just like we rely on Google Search.
Google Search also contains lot of blog posts that are untrue. Do they kill a lot of people? No, Right?
Similarly, AI can be used for basic guidance. After that one can consult psychiatrist if he/she is having money in their pocket.
"It won't happen to me."
But it could! You may feel it makes sense. And for some of us it may make sense too.
What happens when it doesn't and you do not know?
But it could! You may feel it makes sense. And for some of us it may make sense too.
What happens when it doesn't and you do not know?
Yes, you are right about it. It can happen to anyone. We need to cautiously develop AI models related to healthcare.
Problem is that practitioners of mental health treatment themselves cause much damage today. This will not change that. Perhaps we're not too far from an LLM causing less damage than the _average_ mental health practitioner – with far more availability not to mention affordability.
That's arguably for regulators to decide though, not technologists who have no exposure to the harm they cause. I can't even say "engineers" because engineers are held to a standard of care [1] that people slinging code are not. The fact that medical practitioners are sometimes wrong is not a legitimate means to wave away AI safety.
In this instance, the solution isn't immature tech (that may [may!] one day be good tech), it's policy that enables healthcare access. You aren't going to fix that with LLMs. But, you know, that's what tech people keep doing. Poor tech solutions to people problems, and sometimes getting wealthy due to luck along the way.
[1] https://docs.acec.org/pub/18803059-a2fd-2d06-cc39-a6d1dd5752...
In this instance, the solution isn't immature tech (that may [may!] one day be good tech), it's policy that enables healthcare access. You aren't going to fix that with LLMs. But, you know, that's what tech people keep doing. Poor tech solutions to people problems, and sometimes getting wealthy due to luck along the way.
[1] https://docs.acec.org/pub/18803059-a2fd-2d06-cc39-a6d1dd5752...
If you’re shocked by lack of push-back, you are in store for a lot more shocks going forward. People that build things and people that gatekeep, regulate, etc… are mostly disjoint sets. They are the two sides to upcoming war. Progress vs safety. The world as a whole is tilting towards progress right now, we’ve seen the limitations to safety.
I meant more in this community specifically, and, after the AI tool meant to fabricate survey results, YCombinator generally.
I see your “first do no harm” but I’ll counter that it is probably better than the alternative, which is to do nothing. And it may lead to people seeking professional help. It’s something for the unfilled gap in between, it’s not a replacement for professional help.
A good LLMs answers should on average still be better than what an average person would come up with
People give bad advice to other people all the time
I'm not asking your average person for therapy or to help diagnose or work through my mental health issues.
This is in fact not “absurdly dangerous”. It may actually be safer than using a biological intelligence for the same purpose. Given that, and the shortage of biological mental health workers, it seems reasonable that many many more would suffer negative outcomes were they not allowed to use this tech. As such, I’d argue that your comment is absurdly dangerous rhetoric, and your license to post on this web forum should be revoked.
Speaking here with some experience of mental health services, professional misdiagnosis and incompetence within the service is so normalised that that wouldn’t be my main concern.
If someone is experiencing mental health, and chooses to open up and have a chat with such as AI, that itself is a positive sign. That the AI could be on demand only makes it more valuable as a tool.
If someone is experiencing mental health, and chooses to open up and have a chat with such as AI, that itself is a positive sign. That the AI could be on demand only makes it more valuable as a tool.
As a person who also has experience with mental health services, I fully agree.
People seem to have this misguided notion that when they tell someone to "seek professional help" that the "professional" they seek will actually know what they're doing. Generally, they don't. Beyond that, even the best therapists make enormous mistakes. And most people don't get the best.
I honestly think that an AI could do better than most therapists, and I'm sure it would be better than most psychiatrists.
People seem to have this misguided notion that when they tell someone to "seek professional help" that the "professional" they seek will actually know what they're doing. Generally, they don't. Beyond that, even the best therapists make enormous mistakes. And most people don't get the best.
I honestly think that an AI could do better than most therapists, and I'm sure it would be better than most psychiatrists.
AI could… but AI safety is a hard problem, and this particular training - I am not sure how it was done but if you just let it rip on some inputs it wont necessarily always be safe.
The professionals are doing educated guessing when it comes to treatment, both medicinally and psychiatrically.
AI should be part of the toolkit of that guessing.
AI should be part of the toolkit of that guessing.
Access to doctors is limited by the AMA which functions as a cabal or traditional guild: controlling the supply of doctors to maintain high salaries by insulating them from supply and demand (while also controlling or at least pretending to control quality).
In my opinion being 100,000x lower cost gives such a system some leeway in being worse and I think most patients would agree.
In my opinion being 100,000x lower cost gives such a system some leeway in being worse and I think most patients would agree.
> In my opinion being 100,000x lower cost gives such a system some leeway in being worse and I think most patients would agree.
I would heavily disagree with this. We had an AI counselor actively encouraging disordered eating and triggering people into relapse on an eating disorder helpline from logic like this.
I would heavily disagree with this. We had an AI counselor actively encouraging disordered eating and triggering people into relapse on an eating disorder helpline from logic like this.
While I understand that some will disagree, it’s not like there’s an objective standard against which we are measuring both the AI and the human counselors to begin with.
Without a standard it’s just anecdotes and with a standard there’s clarity on when the increased costs no longer justify the lower access and potentially higher quality. I suspect that if we instituted an objective standard we would find that AI is already capable of meeting or exceeding some proportion of people currently employed in the role today.
Without a standard it’s just anecdotes and with a standard there’s clarity on when the increased costs no longer justify the lower access and potentially higher quality. I suspect that if we instituted an objective standard we would find that AI is already capable of meeting or exceeding some proportion of people currently employed in the role today.
This cannot possibly be the case when there was a clear A/B test scenario here. AI was dropped in, nothing else changed about the website, suddenly cases of relapse and worsening conditions went up.
I think you're confusing my statement that "it could be done well" with "your team did it well"
And I’m very familiar with AMA-certified (TM) “doctors” who have done the same.
In this case there’s a fairly clear A/B test where if we dropped in an AI instead of hired counselors, the number of negative outcomes went up.
I've read that it's largely the residency system (which, from the outside, feels more like hazing than training) that limits positions, and I don't think that's under any direct control by the AMA. Wikipedia says it's mostly funded by Medicare.
The AMA lobbies to keep the supply of residency slots restricted* in 1997 the AMA successfully lobbied to freeze the funding for new residency slots at 1996 levels. Lobbying by the AMA is also no small part of why it's so hard for a foreign-born doctor to practice medicine in the US.
*although in the face of the incredible pressure they're about to face from AI they may be opening up
*although in the face of the incredible pressure they're about to face from AI they may be opening up
The residency system is insane, iirc was founded by a guy who was constantly on coke and then took heroin to sleep. Then that became the standard.
It’s an incredibly dangerous and unethical practice that is universal in US hospitals.
I did see one attempt[0] at creating a doctor from an LLM on Github. From their ReadMe.MD overview: "This is an open-source project with a mission to provide everyone their own private doctor"
Hard not to agree with your edit0 as well; search engines often give doomsday diagnosis, how long before we're all hypochondriacs under the latest LLM cult?
[0] https://github.com/llSourcell/DoctorGPT
Hard not to agree with your edit0 as well; search engines often give doomsday diagnosis, how long before we're all hypochondriacs under the latest LLM cult?
[0] https://github.com/llSourcell/DoctorGPT
My biggest concern with that would be that very quickly health authorities or insurance companies will force these on people and you'll be forced to interact with an LLM and not get to talk to a doctor for most stuff.
Zero chance with regulation. We still don’t let machines final read ECGs and insurance companies pay a physician (typically a cardiologist) to overread them. These have been FDA approved algorithms for over a decade already.
I don't see why it would be the worst thing as a first level thing to interact with, freeing up doctors for more important tasks or trickier individuals.
How many people are showing up to doctors for bullshit, going to er for colds and mild flu's, or symptoms that can't be talked until tests are ordered
Last time I went to the Dr, I waited two hours, I finally saw a nurse who took down my symptoms, ordered a test, and said a Dr will follow up,who just gave me some antibiotics for a couple of days before even taking the test.
Why couldn't an llm take my symptoms, match it with similar tests needed and order the test automatically and have me leave with the same antibiotic?then follow up with a doctor when the lab is done.
Same for yearly physicals, just have an llm order all the tests and I'll talk with the dr later.
All sorts of low level stuff can be automated away.
How many people are showing up to doctors for bullshit, going to er for colds and mild flu's, or symptoms that can't be talked until tests are ordered
Last time I went to the Dr, I waited two hours, I finally saw a nurse who took down my symptoms, ordered a test, and said a Dr will follow up,who just gave me some antibiotics for a couple of days before even taking the test.
Why couldn't an llm take my symptoms, match it with similar tests needed and order the test automatically and have me leave with the same antibiotic?then follow up with a doctor when the lab is done.
Same for yearly physicals, just have an llm order all the tests and I'll talk with the dr later.
All sorts of low level stuff can be automated away.
“Bullshit” is almost always a post
hoc diagnosis. In the last 2 years I’ve seen 3 20-somethings with a few days of flu symptoms having used a virtual walk in clinic service before coming to the ER (this is Canada where primary care is mostly non-existent) that had APL (highly treatable) and ended up dying within a few hours from DIC and intracranial hemorrhage (not treatable and how they ended up in my neck of the woods).
There’s still something to be said for the physical exam and eyeball test.
Obviously there’s a large element of survivorship bias at play here but I don’t believe the solution for poor primary care is to accept it will always be shitty and substitute a LLM.
There’s still something to be said for the physical exam and eyeball test.
Obviously there’s a large element of survivorship bias at play here but I don’t believe the solution for poor primary care is to accept it will always be shitty and substitute a LLM.
I didnt mean it as primary care sucks so switch in an llm becasue it sucks as much.
But as away to free up doctors from low effort stuff, churn through lines quicker and get people who actually need focused attention the time with a Dr they need.
Maybe your right and nothing is truly low effort
But as away to free up doctors from low effort stuff, churn through lines quicker and get people who actually need focused attention the time with a Dr they need.
Maybe your right and nothing is truly low effort
You’re right that the vast majority of the time it’s low effort but it’s hard to identify those cases with high specificity up front.
The issue in medicine is there is huge class imbalance, 90%+ of encounters are essentially “negative” or normal. It’s easy for something/someone to look accurate or safe because of pretest probability. The hard part is getting above 90% and why we spend so much time in medicine training.
I hate the word but there’s something in medicine called “clinical gestalt” which is the overall impression one has from certain things in history and exam and doesn’t fit into a decision rule or algorithm, until we find a solution for that I’m not keen on adopting something that distances the patient from the physician.
We’ve tried that with mid levels at my hospital and it didn’t really work out.
The issue in medicine is there is huge class imbalance, 90%+ of encounters are essentially “negative” or normal. It’s easy for something/someone to look accurate or safe because of pretest probability. The hard part is getting above 90% and why we spend so much time in medicine training.
I hate the word but there’s something in medicine called “clinical gestalt” which is the overall impression one has from certain things in history and exam and doesn’t fit into a decision rule or algorithm, until we find a solution for that I’m not keen on adopting something that distances the patient from the physician.
We’ve tried that with mid levels at my hospital and it didn’t really work out.
Discussed here:
https://news.ycombinator.com/item?id=37105352
https://news.ycombinator.com/item?id=37105352
Your black and white opposition to AI-based talk therapy is analogous to being opposed to self driving cars. Yes, the best human driver is better than AI, but what about the worst? Or someone tired or preoccupied with an emotional event in their life? Maybe AI should never drive in congested cities, but what about long-haul trucks on limited access freeways? What about that 'AI' PID loop autonomously operating the gas when you enable cruise control?
I think soon, if not already, heavily tested LLMs will be better than most mental health practitioners for some individuals and conditions. Especially as a first-line intervention, AI could be great at evaluating a patient and placing them with the best suited therapist. Recommendation engines are pretty advanced now, I'd love a therapist match-maker.
I think soon, if not already, heavily tested LLMs will be better than most mental health practitioners for some individuals and conditions. Especially as a first-line intervention, AI could be great at evaluating a patient and placing them with the best suited therapist. Recommendation engines are pretty advanced now, I'd love a therapist match-maker.
... and yet an LLM might be a LOT more accessible and trustworthy (meaning that what you discuss stays private) than a mental health professional.
Even for medical services, in most of the world people actually have very limited access to medical professionals. Would this LLM beat people just attempting to fix things themselves?
Even for medical services, in most of the world people actually have very limited access to medical professionals. Would this LLM beat people just attempting to fix things themselves?
> trustworthy (meaning that what you discuss stays private)
> in most of the world people actually have very limited access to medical professionals
Licensed and competent professionals are also hard to come by. An incompetent psychotherapist can absolutely make things worse, while LLM will just give "average good" advice.
LLMs could be superior when a long, intensive session is needed. You could spend a weekend talking to it and resolving some internal issues.
> in most of the world people actually have very limited access to medical professionals
Licensed and competent professionals are also hard to come by. An incompetent psychotherapist can absolutely make things worse, while LLM will just give "average good" advice.
LLMs could be superior when a long, intensive session is needed. You could spend a weekend talking to it and resolving some internal issues.
It might result in negative outcome for 1 person whereas it can have positive outcomes for 99 people. And there'd be a similar ratio in the real-world psylogical/medical advice anyway: it's not 100% positive for everybody in actual advice too.
As long as it's clearly labeled as this is not medical advice, it's much better than not having it.
As long as it's clearly labeled as this is not medical advice, it's much better than not having it.
I agree with you! We must encourage developers and ML engineers to come up with solutions like these.
Medical costs are too expensive nowadays. Even health insurance won't cover all the costs. We need solutions that can help office workers or students to talk to someone about their mental stress without having to go through the hassle of visiting a doctor and pay an exorbitant fee. If suggesstion provided by these models are atleast 70% reliable, still it's better. You can have an AI companion with whom you can chat and discuss about your problems.
AI won't judge you. AI will listen to your problems patiently. And like your friend whose advices are not 100% reliable. Advices from AI assistant who can be your virtual friend won't be 100% reliable. But, still you will have someone talk to.
Think about people who don't have friends or family. Whom they should to connect to? AI can be atleast their virtual friend and advisor, atleast not 100% correct. But, even your friends are not 100% correct too.
Medical costs are too expensive nowadays. Even health insurance won't cover all the costs. We need solutions that can help office workers or students to talk to someone about their mental stress without having to go through the hassle of visiting a doctor and pay an exorbitant fee. If suggesstion provided by these models are atleast 70% reliable, still it's better. You can have an AI companion with whom you can chat and discuss about your problems.
AI won't judge you. AI will listen to your problems patiently. And like your friend whose advices are not 100% reliable. Advices from AI assistant who can be your virtual friend won't be 100% reliable. But, still you will have someone talk to.
Think about people who don't have friends or family. Whom they should to connect to? AI can be atleast their virtual friend and advisor, atleast not 100% correct. But, even your friends are not 100% correct too.
I think characterizing it as a "conversational" dataset could be seen as misleading in this area. Without reading about it, I'd expect it to be based on transcripts of therapy or intake sessions. I don't think wiki or FAQ websites are quite the same.
I have pre-processed the dataset in a generic conversational format. It's not just raw scraped QnA from Wiki or FAQ's. For few questions which seemed sensitive, I added recommendations to consult a professional mental health care. Transcripts of therapy won't be reliable here because they might contain PII's, unwanted dialogues etc.
Either way, this is a good one for debate given the use is assumed to require careful attention to outputs. Giving someone with a mental health condition advice is fraught with peril, even in a clinical setting conducted by humans with vast experience.
The transcripts of therapy sessions would be quite helpful for improved training of the model, given they contain logic that may not be present in the limited dataset provided. It would be a hope that these detailed interactions would provide improvement into the model's problem solving capacity for helping those with mental illness. As an example, for certain conditions the model may use a more cautious approach to investigation of the source of trauma.
That's not to say therapy session transcripts should be used for prompt tuning after training, which exposes the data directly to the inference pipeline. However, we do know fine tuning the prompts with data is certainly useful for grounding, at the very least.
I'm making the argument that using sensitive data in training is exactly what makes the model better, not filtered data that lacks the wide variety of "expertise" and "experience" that is contained in more sensitive datasets.
If this were done, we can all reasonably assume that "sensitive data" will make its way into the tensors of the model, but the question is whether or not that information is more valuable for everyone's general use compared to the risk of outputting something that diminishes the value for an individual.
I'm not proposing this is a good idea to do, but thinking about it is certainly worthwhile.
The transcripts of therapy sessions would be quite helpful for improved training of the model, given they contain logic that may not be present in the limited dataset provided. It would be a hope that these detailed interactions would provide improvement into the model's problem solving capacity for helping those with mental illness. As an example, for certain conditions the model may use a more cautious approach to investigation of the source of trauma.
That's not to say therapy session transcripts should be used for prompt tuning after training, which exposes the data directly to the inference pipeline. However, we do know fine tuning the prompts with data is certainly useful for grounding, at the very least.
I'm making the argument that using sensitive data in training is exactly what makes the model better, not filtered data that lacks the wide variety of "expertise" and "experience" that is contained in more sensitive datasets.
If this were done, we can all reasonably assume that "sensitive data" will make its way into the tensors of the model, but the question is whether or not that information is more valuable for everyone's general use compared to the risk of outputting something that diminishes the value for an individual.
I'm not proposing this is a good idea to do, but thinking about it is certainly worthwhile.
So to be clear, it's good to see this kind of corpus/dataset.
Just from professional experience, the way patients (and providers) approach discussion of issues is really different from the wiki/faq type-text. It tends to be much more idiosyncratic, sometimes indirect because patients don't recognize patterns, and is more "raw" and unfiltered.
The privacy issues are huge to be sure though. I have done some related research on natural language modeling and understand it's difficult or impossible to separate out the identifying information from the rest of it, and gets worse as an information problem as size increases.
I just personally might refer to the nature of this type of language dataset differently. I'm not sure what the right way of referring to it is though. I might just drop "conversational" or substitute "question" or "information" or something like that. I suppose in the end it doesn't matter much though — people will figure out what it is.
Just from professional experience, the way patients (and providers) approach discussion of issues is really different from the wiki/faq type-text. It tends to be much more idiosyncratic, sometimes indirect because patients don't recognize patterns, and is more "raw" and unfiltered.
The privacy issues are huge to be sure though. I have done some related research on natural language modeling and understand it's difficult or impossible to separate out the identifying information from the rest of it, and gets worse as an information problem as size increases.
I just personally might refer to the nature of this type of language dataset differently. I'm not sure what the right way of referring to it is though. I might just drop "conversational" or substitute "question" or "information" or something like that. I suppose in the end it doesn't matter much though — people will figure out what it is.
Author's writeup about the project: https://medium.com/@iamarunbrahma/fine-tuning-of-falcon-7b-l...
Turning on reader view to read medium.com. How the turntables have ...
Surprised to see 172 items in training set. Is that sufficient scale for QLoRA? I had kinda assumed one needed 1000s
That blew my mind as well. I would have thought a much larger dataset would have been required.
If they are very good examples then less is more https://arxiv.org/abs/2305.11206
arguably it takes more effort/expertise to curate those examples than just pull in a bunch of data that roughly loosely covers the intended surface area. bitter lesson redux?
Most QLoRAs only target attention layers. This gives pretty good performance for minimal processing time. But this one adds adapters to all linear layers. Also I'm not sure what a "step" is but they ran the whole dataset through at least twice.
Here are some of my perspectives on how mental health chatbot can be helpful:
Chatbots can provide immediate support to a large number of people at any time, making mental health resources more accessible to those who might not have easy access to traditional therapy.
Some individuals may feel more comfortable discussing their mental health with a chatbot due to the anonymity it offers, allowing them to open up about sensitive topics without fear of judgment.
Conversational AI can contribute to normalizing conversations around mental health. When a widely used technology addresses these topics, it can help reduce the stigma associated with seeking help for mental health issues.
Mental health struggles can arise at any time. Having a chatbot available 24/7 ensures that support is available even during non-business hours or emergencies.
Chatbots can provide immediate support to a large number of people at any time, making mental health resources more accessible to those who might not have easy access to traditional therapy.
Some individuals may feel more comfortable discussing their mental health with a chatbot due to the anonymity it offers, allowing them to open up about sensitive topics without fear of judgment.
Conversational AI can contribute to normalizing conversations around mental health. When a widely used technology addresses these topics, it can help reduce the stigma associated with seeking help for mental health issues.
Mental health struggles can arise at any time. Having a chatbot available 24/7 ensures that support is available even during non-business hours or emergencies.
The term 'chatbot' has become hopelessly conflated with LLMs in only the last two years, but they're not the only 'chatbot' frameworks in town. Nor are they even properly chatbots in the sense that they can do real dialogue management, slot filling, intent classification, and reliable API calling.
An actual chatbot framework like dialogflow or rasa would be more appropriate here. Even if the conversations feel more artificial, you have drastically more control over the flow of the conversation and the content of the responses.
It's pitiful. The collective amnesia this community has.
An actual chatbot framework like dialogflow or rasa would be more appropriate here. Even if the conversations feel more artificial, you have drastically more control over the flow of the conversation and the content of the responses.
It's pitiful. The collective amnesia this community has.
It seems I’m only one of a few people who finds this area to be promising. Given that mental health care is expensive, that many people suffering are isolated with no one to talk to, and that talking to someone about your issues, whether or not they are a medical professional, is beneficial, I don’t see why this shouldn’t be pushed further. In terms of risk, as others have said, the Internet already exists filled with bad advice and TV has tons of poor medical content. Our apps and media have been hyper optimized by commercial interests in ways that are bad for mental health. Building these resources is the right thing to do, in my opinion.
Tell that to the shareholders when your mental health bot tells a customer how to end their life. Maybe they intentionally jailbroke it because they wanted it to go off script, or it just hallucinated. Not going to matter very much when the stock craters.
This is a GitHub repo, not a NASDAQ corp. Shareholders’ bottom line will never be my concern in the mental health space. If someone wants to build a startup in this space, they should organize it as a public benefit corporation, non-profit, or a co-op.
In terms of outcomes, many people are put on hold calling the suicide hotline. In terms of pranks, the suggested one is criminal and would be prosecuted as such. In terms of accidents, yeah, if this were a product, it needs more fineprint and guardrails.
In terms of outcomes, many people are put on hold calling the suicide hotline. In terms of pranks, the suggested one is criminal and would be prosecuted as such. In terms of accidents, yeah, if this were a product, it needs more fineprint and guardrails.
Well, Headspace is doing it, and I'm pretty sure they're public. I think Ginger gave it a shot a year or two ago. Could go on.
edit: Crunchbase says Headspace acquired Ginger recenty. Didn't realize Ginger was a YC company! Shame. Now I know why they wanted me to sign an NDA before talking to a recruiter.
edit: Crunchbase says Headspace acquired Ginger recenty. Didn't realize Ginger was a YC company! Shame. Now I know why they wanted me to sign an NDA before talking to a recruiter.
[deleted]
I liked this project. But, could you please tell me what should we do if there is out of memory error due to memory issues?
PLEASE do not do this. The field is far too new and the risk matrix of hallucinations are through the roof, ESPECIALLY with those who have mental illness.
This is the sort of stuff that might get legislation passed on open source models or turn public opinion against them.
Yes the USA's most powerful union is the AMA and yes they're the reason # of doctors are low and doctor salaries are skyhigh (especially compared to doctors in the rest of the world), but this is not the way to address the problem, at least not yet!
This is the sort of stuff that might get legislation passed on open source models or turn public opinion against them.
Yes the USA's most powerful union is the AMA and yes they're the reason # of doctors are low and doctor salaries are skyhigh (especially compared to doctors in the rest of the world), but this is not the way to address the problem, at least not yet!
Not only that, but the model was trained on 172 training examples - and the conversations aren't remotely like the therapist conversations I've had in terms of nuance or topic.
This is a toy project. Hence, large-scale collection of data and pre-processing them into conversational format, also removing sensitive information is not possible. I was able to curate only 172 rows of data.
I have shared a notebook and explained detailed steps in my blog - https://medium.com/@iamarunbrahma/fine-tuning-of-falcon-7b-l.... If you are interested to replicate the steps on medical-domain therapy chat transcripts, you are definitely welcome. If you face any issues during fine-tuning steps, you can connect with me on my blog. Would love to help out!
I have shared a notebook and explained detailed steps in my blog - https://medium.com/@iamarunbrahma/fine-tuning-of-falcon-7b-l.... If you are interested to replicate the steps on medical-domain therapy chat transcripts, you are definitely welcome. If you face any issues during fine-tuning steps, you can connect with me on my blog. Would love to help out!
It is interesting. Sorry to pour so much cold water on the topic, it's more the application than the code that people have issue with.
I wonder if retrieval augmented generation would be appropriate here too. Fast, scalable. Fine-tuning is comparatively pretty expensive.
Thanks for sharing your code, iamarunbrahma.
I wonder if retrieval augmented generation would be appropriate here too. Fast, scalable. Fine-tuning is comparatively pretty expensive.
Thanks for sharing your code, iamarunbrahma.
The purpose of this project was to showcase how can we fine-tune large models on a free-tier GPU provided by Colab. Hence, any individual can utilize parameter-efficient fine-tuning methods to tune LLMs on domain-specific datasets.
RAG and fine-tuning serves a slightly different purpose. Fine-tuning helps LLM in learning a new task/skill such as question/answering task, summarization task etc, and improving reliability at producing a desired output such as JSON format structure thereby reducing dependency on prompt engineering.
On the other hand, RAG provides you with external domain-specific knowledge, which one can leverage to get latest information.
RAG and fine-tuning serves a slightly different purpose. Fine-tuning helps LLM in learning a new task/skill such as question/answering task, summarization task etc, and improving reliability at producing a desired output such as JSON format structure thereby reducing dependency on prompt engineering.
On the other hand, RAG provides you with external domain-specific knowledge, which one can leverage to get latest information.
I see, thank you for the explanation. RAG is also appropriate for question/answering but I do see your point. Can see myself returning to your code for inspiration soon.
> I was able to curate only 172 rows of data
Skill issue - focus more on data integrity & less on HN clout
Skill issue - focus more on data integrity & less on HN clout
OP is sharing a personal project, he's not claiming he's solved the mental health crisis.
This is Hacker News. These types of projects are half of the reason I come here, what's with the hate?
This is Hacker News. These types of projects are half of the reason I come here, what's with the hate?
Oh!! So, you want every individual who post their github repo on HN; to collect dataset like billions of dollars funded startups and make a reliable chatbot which not a single well funded startup or listed NASDAQ companies could have done. I salute you.
It’s just Q&A. To call this a chat is very misleading.
Here is another perspective.
The web is already FULL of bad medical information. Older folks treat it as gospel already. All this technology does is make both good and bad information more accessible.
More importantly, there is NOTHING anyone can do to stop this. Nothing the US government can do will stop the proliferation of AI models that do everything including giving medical advice (or other things that we know it will screw up)
"Dr. P<hil>" is still on TV.
I take a pragmatic view on these things - assuming appeals to morality and ethics will fail, what do we do to educate people about this?
The web is already FULL of bad medical information. Older folks treat it as gospel already. All this technology does is make both good and bad information more accessible.
More importantly, there is NOTHING anyone can do to stop this. Nothing the US government can do will stop the proliferation of AI models that do everything including giving medical advice (or other things that we know it will screw up)
"Dr. P<hil>" is still on TV.
I take a pragmatic view on these things - assuming appeals to morality and ethics will fail, what do we do to educate people about this?
Pretty cynical take. The world is already shit, what's another turd on the pile?
Perhaps the right take is to take every experiment to its extreme even if it's confusing.
People have mostly learned photos can be edited. What we need are outlandish experiments and examples, and then media showing off how wild this technology is.
Attempting to bubblewrap this tech both won't work and will lead to more confusion.
People have mostly learned photos can be edited. What we need are outlandish experiments and examples, and then media showing off how wild this technology is.
Attempting to bubblewrap this tech both won't work and will lead to more confusion.
This is why we need pocket universes in which we can spin up and test new technology, like in the peripheral. Too risky to try chaos engineering in a legally risky domain like this
[deleted]
Thank goodness this is the topic of the top comment. Just reading the headline gave me a cold sweat in all the ways this could go wrong.
I get people wanting to be FIRST in a space, but they seem to continuously be the people least needing to be first. Will this ever be something viable? maybe, but it definitely is not now. Yes, it can't get there without going through the growing pains, but let's work those pains out in less detrimental areas first. Areas with much less fallout when it does go wrong
I get people wanting to be FIRST in a space, but they seem to continuously be the people least needing to be first. Will this ever be something viable? maybe, but it definitely is not now. Yes, it can't get there without going through the growing pains, but let's work those pains out in less detrimental areas first. Areas with much less fallout when it does go wrong
This comment just reeks of gatekeeping and ludditism.
How do you expect something to go from “toy” to “viable” if people are not stepping up to build upon the body of knowledge? A lot of people who have created innovative solutions were likely, by your definition, to be among the “least needed” people in a space.
Maybe you would benefit by trying to understand why a harmless toy and proof-of-concept is so threatening.
How do you expect something to go from “toy” to “viable” if people are not stepping up to build upon the body of knowledge? A lot of people who have created innovative solutions were likely, by your definition, to be among the “least needed” people in a space.
Maybe you would benefit by trying to understand why a harmless toy and proof-of-concept is so threatening.
maybe not testing with actual human's fragile mental well being with something being developed as toy, eh?
the words toy and mental health should never be mixed together. this is exactly why it is threatening. someone cavalier enough to think that a group of computer programmers throwing together a chatBot to interact with mental health is just, well, mental.
the words toy and mental health should never be mixed together. this is exactly why it is threatening. someone cavalier enough to think that a group of computer programmers throwing together a chatBot to interact with mental health is just, well, mental.
Literally no one is advocating for this as is to be used as a substitute for currently proven treatment. It shouldn’t mean that we can’t try out different solutions, nor will the final hypothetical product derived from this likely even take the same form or possess the same chat interface.
We cannot let fear stall innovation because of someone might use it incorrectly.
We cannot let fear stall innovation because of someone might use it incorrectly.
So says every person that has ever had a thing they made used in a way never intended. Deny users will do something you've never thought about at your own peril. Unfortunately, in cases like this your peril can have very negative consequences.
This isn't a case of "manning up" and doing something scary. We're not astronauts going to the moon or any other situation where the decisions made will only affect you. This is a matter that when the product is misused or has a "glitch", it is not the maker that suffers.
The fact that you will not even acknowledge this just proves maybe you are not the right person for this.
This isn't a case of "manning up" and doing something scary. We're not astronauts going to the moon or any other situation where the decisions made will only affect you. This is a matter that when the product is misused or has a "glitch", it is not the maker that suffers.
The fact that you will not even acknowledge this just proves maybe you are not the right person for this.
It's the 7B model, which is understood to be a toy model for fun experimentation across the industry and even hobbyists.
I'm sure OP is not going to go out and treat patients with this. 7B barely produces sensical sentences most of the time.
I'm sure OP is not going to go out and treat patients with this. 7B barely produces sensical sentences most of the time.
people are already doing this on Character.ai and other popular AI consumer products, whether the companies want them to or not. I think it's better to accept people want something here and make it good & transparent vs trying to avoid it... it's like alcohol, sometimes people want a thing badly enough they don't care if its bad for them
[deleted]
Edit0: Problem coming from the possibility of the LLM giving bad advice leading to a negative outcome for the person seeking service.