Open source Background Remover: Remove Background from images and video using AI(github.com)
github.com
Open source Background Remover: Remove Background from images and video using AI
https://github.com/nadermx/backgroundremover
96 comments
Are you trying to just cut out the person with those? Looks like adobe and remove.bg do the best on getting a good selection (edges) but OP's tool does best on selecting just the person.
I am just comparing default output from each tool. OP's tool asks for what object to detect and keep, I had selected human so I guess that's why it only kept the person. But yes, my intention was to keep just the person. For this photo, I would go with Adobe because OP's tool adds a halo like outline all around the person. It would differ from photo to photo I think.
Makes sense, and agreed, adobe's has the best cutout and the plant "artifact" is much easier to get rid of than the halo.
Photoroom claim they have the best model but not sure whether that's accurate. It's definitely pretty impressive:
https://twitter.com/matthieurouif/status/1647281207023992834...
Author of project here. Waking up and seeing my project on the front of HN blew me away, thank you everyone. I will go ahead and update the about us on the page on the website.
FYI... the golden value in FG.BG separation is the single pixel bleed between the two. Unless the new background is of the same value as the old, then this would show up as a light or a dark fringe. At erode filter can remove this in hard-edged objects, but an erode would destroy fuzzy objects like hair. This is why compositors often separate soft and hard selections.
This project looks promising!
Your pricing is also attractive, at about half of Unscreen's API. Plus you're open source which is another bonus point.
However, your pricing structure is confusing to me: what does "commercial license for 10" mean when pricing for developers? If we want to integrate your tool in an app, then we would need to have commercial licenses for all generated content, in order to pass that on to the end user. Can you clarify that point?
Also, your outputs all seem to have all-white backgrounds. Would it be possible to specify an image or hex color code that would be used as the new background?
However, your pricing structure is confusing to me: what does "commercial license for 10" mean when pricing for developers? If we want to integrate your tool in an app, then we would need to have commercial licenses for all generated content, in order to pass that on to the end user. Can you clarify that point?
Also, your outputs all seem to have all-white backgrounds. Would it be possible to specify an image or hex color code that would be used as the new background?
Thanks for pointing out how confusing the pricing page was for developers. I went ahead and fixed that. In regards to the background color, it should come out clear. Depending on your viewer I guess it's white. I can put those features on the road map. Have something similar for videos already, so I presume you mean images.
I am a painter who has collaborated with computer vision engineers on style transfer between paintings (aesthetic exemplars) and photographs. One of the things that distinguishes a painter from a camera operator is their depth aware organisation of colour contrasts. I use plural as there are many forms of colour contrast. I believe that effective Foreground-background separation has greatly underestimated potential to impact the future of the photographically acquired image. More power to you sir.
Wow!
I just tested with a complex image and it did a great job. I remember when we had to do this by hand. Thank you!
Works awesome!!! I went to the web site it powers:
https://BackgroundRemoverAI.com
Uploaded this photo:
https://ibb.co/9bv0662
Selected the option to remove all background elements and nearly instantly got this result:
https://ibb.co/WfFsbqW
While, arguably, the legs are part of Zuckerberg, it is an unusual pose and instead the AI picked the subject in a way that is totally suited for inclusion against another background (for example a montage of various founders). The outline it chose looks great.
It's a pleasing result that shows the power of AI.
https://BackgroundRemoverAI.com
Uploaded this photo:
https://ibb.co/9bv0662
Selected the option to remove all background elements and nearly instantly got this result:
https://ibb.co/WfFsbqW
While, arguably, the legs are part of Zuckerberg, it is an unusual pose and instead the AI picked the subject in a way that is totally suited for inclusion against another background (for example a montage of various founders). The outline it chose looks great.
It's a pleasing result that shows the power of AI.
I was going to say that I like the MediaPipe Selfie Segmentation model for doing this sort of thing in a web page, but I've just noticed (when getting the GitHub link[1]) that Google have marked the code as legacy[2] ... no idea if the new solution is better/easier to use[3].
For what it's worth, my CodePen using the old model is here: https://codepen.io/kaliedarik/pen/PopBxBM
[1] - https://github.com/google/mediapipe/blob/master/docs/solutio...
[2] - "Attention: Thank you for your interest in MediaPipe Solutions. As of April 4, 2023, this solution was upgraded to a new MediaPipe Solution."
[3] - https://developers.google.com/mediapipe/solutions/vision/ima...
For what it's worth, my CodePen using the old model is here: https://codepen.io/kaliedarik/pen/PopBxBM
[1] - https://github.com/google/mediapipe/blob/master/docs/solutio...
[2] - "Attention: Thank you for your interest in MediaPipe Solutions. As of April 4, 2023, this solution was upgraded to a new MediaPipe Solution."
[3] - https://developers.google.com/mediapipe/solutions/vision/ima...
Mediapipe is such a mess (but so useful). I wish someone would just extract the main tflite models and make a light wrapper around it on GitHub somewhere
I do realize it's not the same at all, but Pixelmator for Mac is a great little image editor that has "Remove background" and more AI-powered features, such as smart up-scaling.
https://www.pixelmator.com/
https://www.pixelmator.com/
In the latest MacOS I noticed "Remove Background" as an option when editing photos in Photos app. It's like magic. AI for everyone.
it’s in many places on macOS and iOS.
The Lock Screen on iOS , safari, photos, keynote, shortcuts etc…
I don’t think there’s API access though which would be really handy.
The Lock Screen on iOS , safari, photos, keynote, shortcuts etc…
I don’t think there’s API access though which would be really handy.
I think this only applies to photos with depth information, right?
That's above my TechnoDolt©® pay grade
Nope. This works on any old picture you can view. Just as the text highlight in a picture is also os wide. Just background AI processing. It’s very neato
Nope. Works with any image in the Photos app.
I don't remember the steps, but its either in a menu or you can press and hold the foreground object to make a "cut out" of it and paste it in a different app. https://youtu.be/qX82kIaRNKA?t=15
I don't remember the steps, but its either in a menu or you can press and hold the foreground object to make a "cut out" of it and paste it in a different app. https://youtu.be/qX82kIaRNKA?t=15
Well both seem to use u2net for removing backgrounds so almost literally the same thing.
It is possible to directly copy an object out of an image in Photos on iOS.
Why is never a binary provided?
Me and 99% of users won't check the code for malicious behaviour so I might as well run a binary from the web.
Me and 99% of users won't check the code for malicious behaviour so I might as well run a binary from the web.
This is more than entitled behavior, it’s downright harmful.
When (not if, when) binaries get trojanned, this causes blame to be directed at the original author, and takes a lot of work to explain that they are not at fault - this has happened in many supposedly reputable download sites including SourceForge, TUCOwS, Download.com and many others (yes, I haven’t used windows in 20 years or so, no idea what the hip new places are)
Say “thank you”, and spend 10 more minutes yourself to set it up (even if compilation takes 5 hours, it’s usually 10 mins to get it started). And then offer it for others, and handle the ricochets when it gets trojanned with no wrong done by you.
If just 20 people adopted such a process, there would be 98% less complaints of this kind.
When (not if, when) binaries get trojanned, this causes blame to be directed at the original author, and takes a lot of work to explain that they are not at fault - this has happened in many supposedly reputable download sites including SourceForge, TUCOwS, Download.com and many others (yes, I haven’t used windows in 20 years or so, no idea what the hip new places are)
Say “thank you”, and spend 10 more minutes yourself to set it up (even if compilation takes 5 hours, it’s usually 10 mins to get it started). And then offer it for others, and handle the ricochets when it gets trojanned with no wrong done by you.
If just 20 people adopted such a process, there would be 98% less complaints of this kind.
How can a binary I provided possibly be trojanned by someone else?
Genuinely curious.
Genuinely curious.
Trojaned installers by download.com were rampant back in the day. They would take your program and wrap it up in a nice little installer wizard and then also stuff a bunch of adware and spyware in there with it
I don't understand what's the harm of having a releases page with a binary and its md5 hash, or how that keeps anyone from just compiling an unofficial binary themselves and adding malware to it.
Anyone not technical enough to compile a binary has to give up trying to use it or risk some unnoficially distributed executable .
Anyone not technical enough to compile a binary has to give up trying to use it or risk some unnoficially distributed executable .
An md5 can be created for the trojaned binary and be posted along with it.
Not to mention that the md5 checksum is a very poor choice for this purpose because of the ease of creating md5 collisions.
Not to mention that the md5 checksum is a very poor choice for this purpose because of the ease of creating md5 collisions.
But not on the official page, right? And there's nothing stopping someone from doing that now is there? I don't see how the original authors providing binaries is less secure than anything else.
The official page can be hacked, and both malware and md5 of the malware can be placed there.
That's the whole point of using a cryptographic signature backed by a web of trust instead of a mere hash.
That's the whole point of using a cryptographic signature backed by a web of trust instead of a mere hash.
Where would the hash be advertised?
Yeah but still hackers can abuse SEO and direct visits to their pages. If you are not careful you might accidentally download a malicious binary.
Sure, but what does that have to do with distributing binaries off Github? Maybe if Bonzie Buddy and IE6 make a comeback but I don't see that happening.
> If just 20 people adopted such a process, there would be 98% less complaints of this kind.
Okay so if 20 people did the same work over and over it would reduce 98% of the complaints.
Contrast that to if the author did the work once, it would reduce 100% of the complaints!
Okay so if 20 people did the same work over and over it would reduce 98% of the complaints.
Contrast that to if the author did the work once, it would reduce 100% of the complaints!
If every author did.
Do they owe you anything?
Do they owe you anything?
Providing a checksum along with the binary singlehandedly solves your concerns, and you can add gpg signatures if you want.
A checksum can be falsified as easily as a binary, and so can a signature. Only if you participate in a web or trust are you theoretically better off... but most people don't, so all such measures do is give a false sense of security.
Besides what beagle3 wrote: Providing binaries for various platforms is more work than you might think and the people who like to do development work are rarely the same people that like packaging and distribution. That's why developer and maintainer are separate persons more often than not.
Had not thought to make a binary for this project. I will look into this
Because it's a Python project (like most AI code) and distributing Python code in easily usable form is an absolute nightmare.
I've not tried it with AI projects, but pyinstaller does a usually pretty solid job of packing up most python projects, and it's pretty simple to get started with.
Will that package all the dependencies in?
I am not a python dev and testing AI stuff in Python made me hate python ecosystem (not the language ) a lot. All this new AI projects are made by enthusiaste, they depend on a specific CUDA version, a specific A,B,D python lib versions. Very often shit does not work anymore and you need to google and hope other person was unlucky before you and posted some commit version of the stuff that still works.
My advice for people that test AI stuff, after you get it working do not update, try if possible to install the new version side by side and see if it works, it saves you the pain to roll back to a good version.
I am not a python dev and testing AI stuff in Python made me hate python ecosystem (not the language ) a lot. All this new AI projects are made by enthusiaste, they depend on a specific CUDA version, a specific A,B,D python lib versions. Very often shit does not work anymore and you need to google and hope other person was unlucky before you and posted some commit version of the stuff that still works.
My advice for people that test AI stuff, after you get it working do not update, try if possible to install the new version side by side and see if it works, it saves you the pain to roll back to a good version.
Better yet make a Dockerfile
I've packaged various AI/ML/PyTorch/TensorFlow things with PyInstaller in the past. It took some hours of initial work, but the result was good. Things might be easier now.
Encouraging users to run some random binary from GitHub is a really good way to spread all kinds of viruses. People should only run binaries from trusted sources.
Compiling and running random git projects isn't that big of a step up.
You're totally right, it's just as bad, maybe even worse because build tools sometimes ask for admin permissions. People should ideally really on some kind of reputation system or get software from trusted vendors.
You probably don't run curl | sudo bash either but other people have different threat models.
I do sometimes, when the source is trustworthy.
My problem is the assumption by authors that the project being installed is the only one on the machine, and the fact that projects get so tied to particular versions of libraries. Therefore installing the PyTorch specified in this version is likely to be injurious to other existing installations, unless you handle it all in a Conda wrapper.
[deleted]
Deliberate barrier to entry.
If you're going to use something that needs a bit of technical skill to operate correctly, you're going to need at least enough skill to get it running.
Roads would be far safer if every car had some facility by which you had to remove and refit some random engine or braking system component correctly before it would start.
If you're going to use something that needs a bit of technical skill to operate correctly, you're going to need at least enough skill to get it running.
Roads would be far safer if every car had some facility by which you had to remove and refit some random engine or braking system component correctly before it would start.
Would roads be safer, or would there be more mechanics? They seem like different skills.
Roads would be safer, because people would understand more about their car that "PUSH BUTAN GO FAST"
Everyone would be too busy wrenching on their cars to drive them
Roads would be more dangerous. I put the work into my car, I'll drive how I want.
Linked somewhere in the middle of the README file: https://backgroundremoverai.com/?lang=en
Chucked this into a Dockerfile, and had a play with it. (Takes a while for torch to download, had to set the pip default-timeout higher)
Seems to work pretty well, there were some images I had where it left a bit of a blur around hair, but overall it worked pretty well on most of the ones I tried!
Chucked in imagemagik as well to do some trimming/colour manipulation - works pretty nicely
Seems to work pretty well, there were some images I had where it left a bit of a blur around hair, but overall it worked pretty well on most of the ones I tried!
Chucked in imagemagik as well to do some trimming/colour manipulation - works pretty nicely
Could you share your Dockerfile please? I was planning to do exactly the same thing with imagemagik as a complement.
Sure, it was a quick test, but it works for me, YMMV
https://gist.github.com/benperiton/2d37ea18377a80f9b9768522d...
https://gist.github.com/benperiton/2d37ea18377a80f9b9768522d...
Might as well open a PR to add your dockerfile.
Nice work.
I like that you have listed all the resources in the repo.
Side note, I checked your website and the about page is empty
https://backgroundremoverai.com/about/
Side note, I checked your website and the about page is empty
https://backgroundremoverai.com/about/
The about page probably had the background on how the site was created, who made it etc.
But then somehow it was removed.
But then somehow it was removed.
What we need is an open source background check software.
Looking forward to comments under every post like "Uhh, nice project but did you know the author was caught making offensive jokes in 2012"
Updated the about us
Fun fact: The demo image is an example of the hand-in-waistcoat gesture[1].
[1] https://en.wikipedia.org/wiki/Hand-in-waistcoat
[1] https://en.wikipedia.org/wiki/Hand-in-waistcoat
You mean the astronaut shot?
Anyone knows how it compares to rembg in speed/quality?
We need a gimp like app with all these integrated.
For the example image (moon landing), couldn't you just remove the background by getting the original source that has a green screen?
(I'm joking, of course).
(I'm joking, of course).
The photo worked great.
The video I tried couldn’t be played or downloaded.
The video I tried couldn’t be played or downloaded.
The video must of not had a object in the beginning? https://backgroundremoverai.com/results/205ef8ad6b0f4cfa8f66...
Alternative: https://www.remove.bg/
Not open-source nor free. So I wouldn't really call this an alternative.
At least https://baseline.is/tools/background-remover/ is free, but not open-source either.
At least https://baseline.is/tools/background-remover/ is free, but not open-source either.
Once you run out of free "credits", they give you a tiny image and you have to pay to get more for for full-res. For a while that was all I had as far as AI bg removal goes and it was quite frustrating.
We use it for the Zapier integration and it works pretty well. Cloudinary has bg removal transformations too but the color seemed to get degraded.
Would be happy to learn of an alternative!
Would be happy to learn of an alternative!
How fast is this? I know I've always craved a "ai-based Magic Wand selector" for GIMP.
Someone will probably write gimp plugin for https://github.com/facebookresearch/segment-anything soon enough, there already is chrome extension https://github.com/kevmo314/magic-copy and figma plugin https://www.figma.com/community/plugin/1234171506421686729/M...
Canva has one, pretty good actually
Can anyone suggest me an open source, free upscaler powered by Deep Learning/AI?
There's a list of upscaling models here, based on your content: https://upscale.wiki/wiki/Model_Database
The easiest one to get started with is probably Real-ESRGAN: https://github.com/xinntao/Real-ESRGAN
The easiest one to get started with is probably Real-ESRGAN: https://github.com/xinntao/Real-ESRGAN
How do you train such models? Where do you get the training dataset?
neat tool. thank you
We will lose an import data source if we remove unwanted backgrounds. Historical, forensic or just interesting.
You assume that the new one would replace the original.
It’s digital, both would exist. This simply makes it easier to do something people have done manually since the invention of the photograph, or even prior with print.
It’s the equivalent of scissors or a razor blade but automated. It’s not even the first ML based solution, and doesn’t do anything that we don’t already do every day for advertising or film.
It’s digital, both would exist. This simply makes it easier to do something people have done manually since the invention of the photograph, or even prior with print.
It’s the equivalent of scissors or a razor blade but automated. It’s not even the first ML based solution, and doesn’t do anything that we don’t already do every day for advertising or film.
It's not a new technique but more accessible and easier to use and it get integrated into the photo apps.
You assume that people keep photos they don't want, especially if they show "flaws" they don't like.
You assume that people keep photos they don't want, especially if they show "flaws" they don't like.
Okay? Again this has been doable for ages.
You’re making an appeal to “authenticity” that hasn’t been true for probably longer than any of us on here have been alive.
You’re making an appeal to “authenticity” that hasn’t been true for probably longer than any of us on here have been alive.
It's not about the possibility but the ease of use.
ChatGPT doesn't do something new too, text were written before, but know it easier and faster for more people.
https://www.inpixio.com/remove-background/
https://express.adobe.com/tools/remove-background
Adobe is by the best for free services which allows downloading full size transparent image. I compared your tool with the other 3. The results can be seen at https://i.imgur.com/RQT28yp.jpg
I will download this and try to build myself. Thanks for making it open source.