Show HN: A side project in medical devices(chrisseaton.com)
chrisseaton.com
Show HN: A side project in medical devices
https://chrisseaton.com/merseyburns/
46 comments
I don't think you should think about the risk in isolation like that.
Yes, you can't reasonably control the risk of iOS not working as intended, and Apple will never guarantee it does work, and I'm not sure there is anything anyone can do about that on a consumer device. So there's a risk there.
But what's the risk of not having the app because you were afraid of the other risk? Doctors getting fluids wrong - and we found in practice they did get them wrong regularly and significantly.
Reduce one big practical risk, but yes introduce one smaller more theoretical risk. And mitigate where reasonably possible.
These aren't official statements on UK app regulation or the NHS's policy on Mersey Burns, though.
Yes, you can't reasonably control the risk of iOS not working as intended, and Apple will never guarantee it does work, and I'm not sure there is anything anyone can do about that on a consumer device. So there's a risk there.
But what's the risk of not having the app because you were afraid of the other risk? Doctors getting fluids wrong - and we found in practice they did get them wrong regularly and significantly.
Reduce one big practical risk, but yes introduce one smaller more theoretical risk. And mitigate where reasonably possible.
These aren't official statements on UK app regulation or the NHS's policy on Mersey Burns, though.
> I don't think you should think about the risk in isolation like that.
I don't disagree with the sentiment but FDA asks you precisely to do this analysis piecewise by what they call "SW units". And for each SW unit, you have to define a severity of hazard that can result from a malfunction of that SW unit. (from minor injury to death or irreparable damage)
Now you can always say you have a single "SW unit" to not have to check the boundaries or interactions, but then all the SW should have been developed with the same methodology/process, and this is not possible when you are using SW coming from an external source. (because you have no idea how it has been developed, and it was likely not targeting medical devices)
As as aside, the recommendation by the FDA is that, prior any mitigation, you have to assume that a SW bug as a 100% chance of happening. (That tells you how confident are the regulators..) But all SW bugs do not necessarily create an hazardous situation.
> But what's the risk of not having the app because you were afraid of the other risk? Doctors getting fluids wrong - and we found in practice they did get them wrong regularly and significantly.
You are allowed to use the benefits risk analysis by the FDA, but you have to demonstrate first there are no other mitigations possible, that you are ALARP (As low as reasonably possible) [0]. You need to explicitly write it in a risk management document to demonstrate you have analyzed that source of hazard.
To be fair, I am not anti regulation, but I think most of these standards are more applicable to SW written in the 80s or 90s when the SW stack where a bit smaller and simpler. And I have hard time to understand how they can be applied in practice in complex SW stacks like iOS. That's why I was interested by your experience, but it seems NHS did not challenge you on the finer implementation details. I understand this is complex because we are talking about 2 different regulation frameworks, so it is not necessarily transferable.
Thanks for chiming in, that is still great to know that is is possible to get something approved on a modern tech stack.
[0] https://en.wikipedia.org/wiki/ALARP
I don't disagree with the sentiment but FDA asks you precisely to do this analysis piecewise by what they call "SW units". And for each SW unit, you have to define a severity of hazard that can result from a malfunction of that SW unit. (from minor injury to death or irreparable damage)
Now you can always say you have a single "SW unit" to not have to check the boundaries or interactions, but then all the SW should have been developed with the same methodology/process, and this is not possible when you are using SW coming from an external source. (because you have no idea how it has been developed, and it was likely not targeting medical devices)
As as aside, the recommendation by the FDA is that, prior any mitigation, you have to assume that a SW bug as a 100% chance of happening. (That tells you how confident are the regulators..) But all SW bugs do not necessarily create an hazardous situation.
> But what's the risk of not having the app because you were afraid of the other risk? Doctors getting fluids wrong - and we found in practice they did get them wrong regularly and significantly.
You are allowed to use the benefits risk analysis by the FDA, but you have to demonstrate first there are no other mitigations possible, that you are ALARP (As low as reasonably possible) [0]. You need to explicitly write it in a risk management document to demonstrate you have analyzed that source of hazard.
To be fair, I am not anti regulation, but I think most of these standards are more applicable to SW written in the 80s or 90s when the SW stack where a bit smaller and simpler. And I have hard time to understand how they can be applied in practice in complex SW stacks like iOS. That's why I was interested by your experience, but it seems NHS did not challenge you on the finer implementation details. I understand this is complex because we are talking about 2 different regulation frameworks, so it is not necessarily transferable.
Thanks for chiming in, that is still great to know that is is possible to get something approved on a modern tech stack.
[0] https://en.wikipedia.org/wiki/ALARP
There is absolutely no way you'll remember this but years and years ago--when I was maybe 13--I found Katahdin somewhere online and was absolutely enamored by the idea (I can say with a lot of confidence that it's what sparked my interest in programming languages/compiler research). I think I contacted you for help installing and using the project. You were one of the nicest people I've ever met for having the patience to assist me with something you had probably put down for a couple of years at that point when I probably understood 0% of what you were saying. Anyways, I remember at the time you had just listed Mersey Burns on your website. It's super cool to see that it's still something that you're doing and that it's been a success. Just wanted to thank you again and congratulate you on this :)
Thanks very much.
Katahdin sort of lives on in Truffle and TruffleRuby https://chrisseaton.com/truffleruby/ - Truffle is a framework for implementing languages that use partial evaluation of ASTs.
Katahdin sort of lives on in Truffle and TruffleRuby https://chrisseaton.com/truffleruby/ - Truffle is a framework for implementing languages that use partial evaluation of ASTs.
Looks pretty cool: https://chrisseaton.com/katahdin/
Also see lambdanative.org, which the authors created explicitly to build medical devices out of low-end mobile devices for third-world-countries. It is cross-platform.
They have several such apps on the website.
Warning: it uses the Scheme programming language.
They have several such apps on the website.
Warning: it uses the Scheme programming language.
We found that there were a lot of awards and prizes that we could enter.
I would love to see a list of these.
Edit: I guess this is a partial list within the article itself, UK based stuff, though I would be more interested in a US based list:
Winner, Excellence in Mobile Healthcare and overall winner, eHealth Insider Awards, 2013
Highly Commended, Improving Care with Technology, Health Service Journal Awards, 2013
Highly Commended, Innovative Mobile App of the Year, BCS UK IT Industry Awards, 2013
Highly Commended, Best Use of Mobile Technology in Healthcare, eHealth Insider Awards, 2012
Winner, Excellence in Innovation, NHS North West Health Innovation Awards, 2011
I would love to see a list of these.
Edit: I guess this is a partial list within the article itself, UK based stuff, though I would be more interested in a US based list:
Winner, Excellence in Mobile Healthcare and overall winner, eHealth Insider Awards, 2013
Highly Commended, Improving Care with Technology, Health Service Journal Awards, 2013
Highly Commended, Innovative Mobile App of the Year, BCS UK IT Industry Awards, 2013
Highly Commended, Best Use of Mobile Technology in Healthcare, eHealth Insider Awards, 2012
Winner, Excellence in Innovation, NHS North West Health Innovation Awards, 2011
Really interesting read! Do you mind sharing the basics of the process you went through to get regulated by the NHS?
It's really a form of self-regulation - you fill in a form - but you need to justify why you think the app will function as you have said it will, and why that's correct in the first place. We built a chain of documentation. Why do we think the hardware does what we coded? Why do we think what we coded was what we designed? Why do we think what we designed matches what we understand about the clinical policy? Why do we think we understand the clinical policy correctly? Why do we think the clinical policy is correct? For each of those questions we provided evidence and argument.
It looks like the process has been refined since we did it
https://www.gov.uk/guidance/register-as-a-manufacturer-to-se...
I'm not a lawyer or expert on regulation so don't take my word for it.
It looks like the process has been refined since we did it
https://www.gov.uk/guidance/register-as-a-manufacturer-to-se...
I'm not a lawyer or expert on regulation so don't take my word for it.
I find this a really impressive project: 1. it actually helps society and 2. the author did it as a side project, not being a software developer, writing an iOS, Android and web app that's polished and being used. Wow! Great write up!
Really cool stuff.
I have few friends who are pharmacists maybe I could develop something with them. :)
I have few friends who are pharmacists maybe I could develop something with them. :)
If they don't have a solution already, anything that schedules custom pharmacy mixes based on physician orders in a medical records system is worth millions. I wouldn't go so far as to connect it to an actual drug mixing system though -- let the pharmacists take care of that.
These are short-lived mixes that can't be stored for a long time, so must be prepared on-site.
Bonus if you can plug into an inventory system and get alerts for drug shortages. These can happen pretty regularly, on the order of six or seven times per year.
You'll write a lot of custom code to connect to that medical system API, and often there are surprise and sparsely documented features. That's mostly why you set a high price for it.
We did one of these and quickly saved the hospital millions because we had before and after photos of pharmacy waste generated. They weren't wasting drug mixes for cancelled orders any more when a doctor forgot to call/fax the pharmacy or the patient had been checked out/moved.
Before-hand, they were collecting this information manually with some cobbled together reports. It was nuts.
This isn't turnkey though. You will work for the money trying to sell to hospitals with bloated change management processes.
These are short-lived mixes that can't be stored for a long time, so must be prepared on-site.
Bonus if you can plug into an inventory system and get alerts for drug shortages. These can happen pretty regularly, on the order of six or seven times per year.
You'll write a lot of custom code to connect to that medical system API, and often there are surprise and sparsely documented features. That's mostly why you set a high price for it.
We did one of these and quickly saved the hospital millions because we had before and after photos of pharmacy waste generated. They weren't wasting drug mixes for cancelled orders any more when a doctor forgot to call/fax the pharmacy or the patient had been checked out/moved.
Before-hand, they were collecting this information manually with some cobbled together reports. It was nuts.
This isn't turnkey though. You will work for the money trying to sell to hospitals with bloated change management processes.
Pharmacist & software dev here -- my goodness please do, the technology infrastructure supporting pharmacy is massively outdated and I've had to fight against it to do my job properly. Among many other problems, explore what can be done in compounding.
Any plans to get this through FDA approval in the US? I don't know which process you'd have to go through(510k, De Novo or PMA) but I'd imagine the NHS approval would help a lot.
We tried it a couple of times, but it was always more serious than we were willing to invest into.
The great thing about our arrangement with the NHS was we gave them the IP, so they owned the app. They were also the only people using the app (the have an effective monopoly on emergency medicine in the UK), so if something went wrong it's the NHS using an NHS app and they could work it out themselves, and not companies suing companies.
I don't think that kind of simple arrangement would be possible in the US, but I don't know much about US healthcare apart from it isn't nationalised.
The great thing about our arrangement with the NHS was we gave them the IP, so they owned the app. They were also the only people using the app (the have an effective monopoly on emergency medicine in the UK), so if something went wrong it's the NHS using an NHS app and they could work it out themselves, and not companies suing companies.
I don't think that kind of simple arrangement would be possible in the US, but I don't know much about US healthcare apart from it isn't nationalised.
Yeah, it was difficult for me to see from the FDA's website how exactly you'd get approved. If you did have to go through De Novo or PMA then you're probably looking at years and years if you can even get the regulators to accept it. It's not like there's a predicate device for what you're doing so there's nothing you can point to. From what I understand, De Novo is where cool ideas go to die.
FDA protects people from a lot of bad devices, but it also makes it really difficult for people to innovate here. My last job was with a company doing a 510k and we were delayed 6 months while the branch of the FDA that handled ventilators tried to figure out how to work with the branch of the FDA that handled oxygen concentrators.
I don't blame you.
FDA protects people from a lot of bad devices, but it also makes it really difficult for people to innovate here. My last job was with a company doing a 510k and we were delayed 6 months while the branch of the FDA that handled ventilators tried to figure out how to work with the branch of the FDA that handled oxygen concentrators.
I don't blame you.
In the US, your company would purchase insurance, and then bake that into the price.
This is really awesome. Can this be made available in India?
Sorry, not without an understanding of how India manages medical device regulation.
Why it is not available in Turkey?
The app constitutes a medical device - we'd have to get it regulated by the Turkish government, and I have no idea how we'd do that.
Could you make another one, where the patient looks like a clown and call it, "a game" ?
Sounds like a terrible idea if you ever plan to sell your product in another market. Regulators aren't typically big fans of trying to deliberately skirt their regulations.
(I work in the medical software industry.)
(I work in the medical software industry.)
Nice idea, but I would imagine there are a few App store terms and conditions that would not allow a medical app dressed as a game. Then again, if it's Steam based...
That sounds like asking for trouble.
> My fiancée was working with a plastic surgeon, Rowan Pritchard Jones, who had also been in the Royal Army Medical Corps, who was thinking about his own medical equivalent of a side-project - writing a book with his colleague, Paul McArthur. They decided to do some form of app instead to make it different, and asked me if I was interested in working with them.
This shows once again that building successful software depends not just on coding skills or UX design skills, but more on the connections we have. I'm betting most senior developers on HN could have come up with this kind of solution (or perhaps even more advanced ones). Connections determine, through a seemingly random process, our ability to actually find interesting stuff to work on. I just wish it was simpler to find interesting project ideas.
This shows once again that building successful software depends not just on coding skills or UX design skills, but more on the connections we have. I'm betting most senior developers on HN could have come up with this kind of solution (or perhaps even more advanced ones). Connections determine, through a seemingly random process, our ability to actually find interesting stuff to work on. I just wish it was simpler to find interesting project ideas.
Yeah. To take it a step further - I would argue that for a lot of things, the coding isn’t most difficult part of the innovation process.
> the coding isn’t most difficult part of the innovation process.
This. Exactly my sentiments. I believe coding could also be one of these easiest things to do. Hardest are - requirements gathering and selling.
This. Exactly my sentiments. I believe coding could also be one of these easiest things to do. Hardest are - requirements gathering and selling.
Funny how everyone thinks other stuff is hard. Me, I think gathering requirements and deciding what to build is the fun part. Following through with the plan, managing the project, coming up with a schedule, making sure everything gets done on time, that's what I consider the hard part.
Great. Then, let's build a team of experts :-)
Both you and the parent seem to assume that there is this clear boundary between gathering requirements, and implementing the solution.
The hard part is accepting that this is an ongoing cycle and that you need to find ways to build inertia and continually evolve the value proposition.
The hard part is accepting that this is an ongoing cycle and that you need to find ways to build inertia and continually evolve the value proposition.
Sorry, I do not have this assumption. Prioritizing the requirements is another big challenge.
A lot of very valuable companies are built around what are little more than CRUD apps, or at least started that way.
Twitter, Facebook, and Reddit started as relatively basic apps, basically glorified message boards.
Twitter, Facebook, and Reddit started as relatively basic apps, basically glorified message boards.
And evolved into glorified Skinner boxes?
Of course not, it's our own biases that lead us to think we are the gods through the expression of a creative process. Because we are human, and cannot see past what's in front of us, we mistake our tools for creating.
The first steam engines were riveted by metal workers who may have felt similarly.
The first steam engines were riveted by metal workers who may have felt similarly.
This is similar to college students who study “consulting” as a discipline. Certainly there are important things to learn about consulting strategies. However, if you aren’t a subject matter expert in the field you are consulting, I don’t want to hear it.
For a lot of consulting disciplines, and management consulting in particular, the only domain expertise you require is repeating back to the customer what they told you and finding a way to manipulate and/or present what they've told you such that it supports the conclusion they would like.
This sounds like sarcasm but I'm pretty sure every management consultant would admit that's what they do.
(technical consultants one hopes are actual domain experts, as you say, though I've often found it to be hit or miss even so).
This sounds like sarcasm but I'm pretty sure every management consultant would admit that's what they do.
(technical consultants one hopes are actual domain experts, as you say, though I've often found it to be hit or miss even so).
Often the point is that there's no single "they". If an organization is sufficiently dysfunctional that group A refuses hear what group B has to say (and many organizations are), then management getting a summary (filtered!) of group B's recommendations from someone they trust is quite valuable.
I've asked a few friends who are management consultants this. Most say around 25-50% of their job is doing that. They don't view this negatively though because in many cases getting a reputable consulting group to tell you what you already know helps the decision actually get implemented. The rest of their time they spend doing things like collecting information that the client can't/doesn't want to do on their own.
Great point, completely agree. It took me several years to understand that the quality of the technology is only a single component (often a very minor one) to making a successful product (let alone a successful company).
Connections and a strong network will improve odds of success but there is a lot of great software that has and will be made be people starting at 0 without a fiancé who has doctor friends.
There are a lot of things that might discourage a young hacker and we shouldn’t add “you don’t have a network” to that list.
Maybe a more positive message would be “it shows how building a network along with apps will increase the likelihood of success and certainly the likelihood of making it to the hacker news front page”
There are a lot of things that might discourage a young hacker and we shouldn’t add “you don’t have a network” to that list.
Maybe a more positive message would be “it shows how building a network along with apps will increase the likelihood of success and certainly the likelihood of making it to the hacker news front page”
And motivation . The exposure to the problem gives you motivation to solve it . If someone randomly gave this idea, one could probably just ignore it ..
I am working in medical devices but I am more familiar with the FDA regulations.
What I wonder regarding risk management is how you justify the safety of QT /iOS / Android. In its risk management, the FDA asks you to list of your SW dependencies you are using "off the shelf", and you have to analyze how any failure of any of your dependencies could lead to a failure of your device/application (which is good sense really).
So usually for a simple library it is easy to justify and identify the failure mode, but you look at the OS or big frameworks, it is always harder to find a clear mitigation. Especially at the OS level where potentially any part of memory could be corrupted, your thread not being scheduled, etc.
For example, in your case imagine you have a driver or QT rendering bug that makes the red overlay layer becoming transparent on some HW. That would lead to the user not identifying the correct area and in the end lead to a wrong computation. (I can see on some screenshots you have the percentage displayed so that could be a mitigation in itself)
I am working in the embedded space so generally we have less dependencies and a simpler code base, but we are always reluctant to use off the shelf SW because of the added risk management required. This is counter intuitive for safety but from a regulatory stand point it is sometimes easier to reimplement some features, than using a proven library, because this is less a hassle to justify or maintain.
So was it something raised by the regulator ? Can you give us some example of how you addressed it?