.NET Core Image Processing(blogs.msdn.microsoft.com)
blogs.msdn.microsoft.com
.NET Core Image Processing
https://blogs.msdn.microsoft.com/dotnet/2017/01/19/net-core-image-processing/
69 comments
Mono shipped a reimplementation of the types in the System.Drawing namespace which uses GDI+ (note the +) on Windows. This reimplementation is also what CoreCompat.System.Drawing is based on. To my knowledge Mono did shim a few Windows APIs, mostly related to window messages so WndProc can be overridden in a useful manner, but not actually GDI, as far as I know.
And the implementation was buggy as hell. When I needed to resize pictures on mono + Linux I finally had to resort to manually building libgb, writing a simple wrapper for the resizing part, and use that, as back then every other alternative as either incomplete, or pain to use and/or buggy.
Note: libgd is a well designed native library with design emphasis on simple interop wrappability.
Note: libgd is a well designed native library with design emphasis on simple interop wrappability.
This makes sense as System.Drawing (aka GDI+) was mainly used for WinForms and has never been supported for web servers. I suspect Mono ported it to support their Windows Forms port.
It may just work on Windows but it won't keep working when used server-side in a web app. MS specifically recommend against using it in ASP.NET. I learned the hard way what happens if you ignore the warnings when I was supporting a site that did this.
Edit: I wrote about image optimization and resizing in chapter four of my book on ASP.NET Core - https://unop.uk/book/
It may just work on Windows but it won't keep working when used server-side in a web app. MS specifically recommend against using it in ASP.NET. I learned the hard way what happens if you ignore the warnings when I was supporting a site that did this.
Edit: I wrote about image optimization and resizing in chapter four of my book on ASP.NET Core - https://unop.uk/book/
I may have to buy that!
Great to see the positive comments here. ImageSharp is my baby and it fills me with a lot of confidence that it's something that developers both want and need.
Why would you expect negative comment? .NET Core is a solid platform for web development. It's easy to deploy and relatively fast compared to the usual Python/Ruby/PHP and co... only matched by Java(controlled by Oracle) and Go(which is obnoxious). .NET Core is growing at a fast pace, and it deserves it, C# and F# are phenomenal languages.
This isn't my first rodeo with Hacker News, I've unfortunately had negative responses before when sharing my work.
Thank you for your work! I used it with a client that had a System.Drawing impl. Just making thumbnails of uploaded images. It actually was our CPU bottleneck! And then it started crashing when load was high, making us drop or reprocess (which added more load and drops...).
With your lib and a few hours of work, all that disappeared and CPU went way down. Thanks!
On a side note, users should try to send gratitude to open source authors in general. I know it's hypocritical of me, since I didn't take the time to email you before this! I've done it a few times in the past and the replies I've received indicated that I was the only one to ever thank them! That sucks. I've written some stuff I know more than a few companies built entire products and services with -- I've received about 2 notes in 6-7 years. It's not much but just a bit of thanks really provided motivation.
Cheers to you.
With your lib and a few hours of work, all that disappeared and CPU went way down. Thanks!
On a side note, users should try to send gratitude to open source authors in general. I know it's hypocritical of me, since I didn't take the time to email you before this! I've done it a few times in the past and the replies I've received indicated that I was the only one to ever thank them! That sucks. I've written some stuff I know more than a few companies built entire products and services with -- I've received about 2 notes in 6-7 years. It's not much but just a bit of thanks really provided motivation.
Cheers to you.
That's great to hear, With future updates and a little help I'm sure we can improve the performance further and make us the g0-to library.
And thanks for saying thanks, it is rare and always very much appreciated.
And thanks for saying thanks, it is rare and always very much appreciated.
[deleted]
It's not exactly difficult to find negative comments for anything on Hacker News.
Cool. I'm assuming you do a lot of matrix operations, so I'm wondering if you've come across the Matrix<> type in Math.Net?; It has plug-in 'providers' for concrete implementations, and there are providers that are backed by Intel MKL, OpenBLAS, ATLAS, CUDA, etc.
If you don't register a provider you just get plain .NET code running serially (though it may user Vector<T> in future versions).
See:
https://github.com/mathnet/mathnet-numerics/tree/master/src/...
If you don't register a provider you just get plain .NET code running serially (though it may user Vector<T> in future versions).
See:
https://github.com/mathnet/mathnet-numerics/tree/master/src/...
Thanks. Didn't know about that library. We do a lot of matrix operations but have so far found System.Numerics.Vectors covers our immediate use cases.
I found that matrix multiplication of dense matrices using MKL was about 60x faster than 'plain' C# code (on a recent core i7 CPU). The Vector<T> class will get you only some of that gain - mainly because MKL uses the FMA CPU instruction (fused multiply and add) that is designed specifically for matrix multiplication, + lots of other optimisations that Intel can make that the .NET jitter can't/won't.
That's some improvement. Will definitely have a proper look.
Great to see a more generic pure-.NET image processing library. When I was doing this 5 years ago we basically had AForge.NET which was focused more on correctness of algorithms than speed. I tried to leverage that, but instead ended up with a totally custom image processing pipeline instead -- oh the battles with the GC and unsafe code...
Out of curiosity what is your overall vision for the library? Are you looking for a System.Drawing high-level API or more of an OpenCV-like low-level API?
Out of curiosity what is your overall vision for the library? Are you looking for a System.Drawing high-level API or more of an OpenCV-like low-level API?
I'd say somewhere in between. I've started with a very high level fluent API since I see that as the most common use case but have exposed a couple of important properties that will allow more low-level operations (pointers etc).
I have also created a generic image pipeline Image<TColor> that uses all the packed pixel color models used in XNA and Monogame which make it useful in gaming for loading textures.
The dream is that eventually the community will help port a lot of the algorithms over from libraries like Accord, OpenCV and AForge to allow usage in more scientific scenarios. Some of them will fit in the fluent API most will not but the tools should already be within the library to allow porting and usefulness.
I have also created a generic image pipeline Image<TColor> that uses all the packed pixel color models used in XNA and Monogame which make it useful in gaming for loading textures.
The dream is that eventually the community will help port a lot of the algorithms over from libraries like Accord, OpenCV and AForge to allow usage in more scientific scenarios. Some of them will fit in the fluent API most will not but the tools should already be within the library to allow porting and usefulness.
My experience with any ImageMagick and derivateives is terrible. Obscure install is needed, app is not portable (which is normally a great default feature of .Net apps), or building it is difficult. Also its security track (of the native parts) is very bad, so i strongly discourage anybody using those libraries.
On the other hand ImageSharp is convinient, and the author is a nice helpful, responsive guy. I totally suggest using his library. When I needed it once for mono, though then it did not meet my needs, Mr. South was really helpful. Eventually I had to stick to manually wrapping libgd, but I'm about to revisit that code and change it to use a new ImageSharp version! (Also the fluent APIs make working with images using that lib pretty awesome!)
On the other hand ImageSharp is convinient, and the author is a nice helpful, responsive guy. I totally suggest using his library. When I needed it once for mono, though then it did not meet my needs, Mr. South was really helpful. Eventually I had to stick to manually wrapping libgd, but I'm about to revisit that code and change it to use a new ImageSharp version! (Also the fluent APIs make working with images using that lib pretty awesome!)
Magiic.net works fine but it occludes the real API so you might struggle to do certain things
The real api is a bit of brainfuck to me.
I'm not sure what psychedelic the creators of ImageMagick were on, or maybe they thought it is kinda funny, but for me the GetMagicWand call was also a single sign that I don't want to use that.
So what I mean by this: in my opinion occluding the real API is probably a feature of Magiic.net. Still I do not like if a library needs systemwide install and config files, etc, as ImageMagick/GraphicsMagic does. Maybe the creators of these libs have solved the portability problems created by this, but I'm disgusted by these native libraries, and I avoid them. (Back then the one wrapper I tried needed a systemwide install of imagemagick. A click through wizard needing administrative rights, instead of dropping a dll into the PATH or the program directory... Also it has config files. Config files.. for a library. IMHO this is insane! It is a good approach to configure apps by config files. Libraries should be configured by API calls, or build time constants.)
Having custom build scripts for 3rdparty code is a huge work to keep up to date which will always lag behind, thus the security will also lag behind. On the other hand this is needed to solve the portability issues of the native parts. I find this approach unproductive, but wish the maintainers the best.
So what I mean by this: in my opinion occluding the real API is probably a feature of Magiic.net. Still I do not like if a library needs systemwide install and config files, etc, as ImageMagick/GraphicsMagic does. Maybe the creators of these libs have solved the portability problems created by this, but I'm disgusted by these native libraries, and I avoid them. (Back then the one wrapper I tried needed a systemwide install of imagemagick. A click through wizard needing administrative rights, instead of dropping a dll into the PATH or the program directory... Also it has config files. Config files.. for a library. IMHO this is insane! It is a good approach to configure apps by config files. Libraries should be configured by API calls, or build time constants.)
Having custom build scripts for 3rdparty code is a huge work to keep up to date which will always lag behind, thus the security will also lag behind. On the other hand this is needed to solve the portability issues of the native parts. I find this approach unproductive, but wish the maintainers the best.
Definitely do please give it a try!
Amazed this is trending. If anyone has any interest in image processing of performance in general please get in touch and give me a hand. Any help will be appreciated.
Nice post. Looks like I'll need to update the compatibility list: https://github.com/jpsingleton/ANCLAFS#image-manipulation
NB: You shouldn't use System.Drawing in a web app. It can lead to memory leaks and lots of pain. Even the newer WPF equivalents are not safe for use on a web server.
NB: You shouldn't use System.Drawing in a web app. It can lead to memory leaks and lots of pain. Even the newer WPF equivalents are not safe for use on a web server.
What would you recommend instead of System.Drawing in a web app?
Any of the ones listed in the article?
Any of the ones listed in the article?
Unless you have a good reason to then I would recommend against rolling your own (with any library). Use an existing cloud service that specializes in it.
Most b2b saas products have a specific reason to not use a third party for everything - their customers don't want their data sprayed everywhere. Add to that anyone with a government contact.
Nice list. Any idea on the status of IronPython?
Is there any road map on getting it into Core?
[OT] I didn't know about this repo. Its so amazing! Thanks so much for maintaining it :)
You're welcome. I'm glad it's useful. It's a community effort so feel free to submit a PR or an issue.
P.S. There used to be a separate website but since GH broke how pages work the domain now just redirects to the repo.
P.S. There used to be a separate website but since GH broke how pages work the domain now just redirects to the repo.
What are the best resources for someone who wants to move from .NET desktop development to .NET web development? All the popular sites like Udacity and FreeCodeBootcamp rarely (if ever) touch the MS stack.
I know this isn't exactly the right place to ask, but I've had no luck anywhere else...
I know this isn't exactly the right place to ask, but I've had no luck anywhere else...
If you are already familiar with .NET, you might find some courses on Microsoft Virtual Academy a good starting point. There is a web development category:
https://mva.microsoft.com/training-topics/web-development#!i...
Looking at the courses available, I would probably start with: https://mva.microsoft.com/en-US/training-courses/getting-sta...
then look into either ASP.NET MVC: https://mva.microsoft.com/en-US/training-courses/introductio...
or ASP.NET Core: https://mva.microsoft.com/en-US/training-courses/intermediat...
This is coming from someone that hasn't actually used these, so take my recommendations with a grain of salt.
Looking at the courses available, I would probably start with: https://mva.microsoft.com/en-US/training-courses/getting-sta...
then look into either ASP.NET MVC: https://mva.microsoft.com/en-US/training-courses/introductio...
or ASP.NET Core: https://mva.microsoft.com/en-US/training-courses/intermediat...
This is coming from someone that hasn't actually used these, so take my recommendations with a grain of salt.
The training resources available on the ASP.net website are excellent and cover all skill levels.
https://www.asp.net
https://www.asp.net
I don't know of any reliable tutorials, but if you want to try jumping into a project, you can download Umbraco and follow some basic 'Getting Started' guides. Some of what you will learn is Umbraco-specific, but you'll also see a lot of how .NET likes to do things for web projects.
Alternatively, you can just get Visual Studio and use their "ASP.NET Web Application" template and poke through what is generated, if you want something a bit smaller to start with.
Links:
Umbraco: https://umbraco.com/
Getting Started Guide: https://our.umbraco.org/documentation/Getting-Started/
Alternatively, you can just get Visual Studio and use their "ASP.NET Web Application" template and poke through what is generated, if you want something a bit smaller to start with.
Links:
Umbraco: https://umbraco.com/
Getting Started Guide: https://our.umbraco.org/documentation/Getting-Started/
Your absolute best bet is to sign up for Visual Studio Dev Essentials and try out all the 3 month trials they give. Pluralsight in particular is a really great resource.
https://www.visualstudio.com/dev-essentials/
https://www.visualstudio.com/dev-essentials/
You need to be specific. Most people doing web on .NET probablly use .NET Asp.NET MVC... I've used it a lot, and it's ok. But there are alternatives. For instance using F# and https://suave.io/ Which I think is really nice.
if you want to learn .Net development then you should try pluralsight. it's a great resource for everything .net. I'm on mobile right now but you can get free 3 months subscription. you can also get a free month I think to wintellect.
Pluralsight is the best training site for MS related technologies, it is how they started, before embracing other stacks.
I have a ton of code that relies on System.Drawing so I've been watching this closely. These are projects that go back over 10 years now. I have code in both web server and client side versions. System.Drawing offers some features that other image processing libraries don't. My use case probably isn't typical but when you are trying to go back and forth from vector to raster data you need access to the point data (along with manipulation) that is going to be drawn. System.Drawing provides this flexibility through the GraphicsPath object. It is good to see some official word on this though.
Chat to us on github. Our path drawing public API's are very similar to System.Drawing so we may well be able to provide you with the same functionality.
I'll check it out James. I keep up with you on twitter, wish I had more time to contribute. I have a lot of respect for your hard work though.
Thanks!
Also take into account that rely on the GDI API won't work in Azure App Services (which is where most new azure based web apps reside) [1]. It would be nice if this reduced some of the issues with 3rd party libraries (pdf generation etc) running inside of app services. We've had some issues having to keep old web/worker roles around for this exact reason and it costs us more money.
>Components rely on GDI API may not work on Azure Websites. The workaround is moving to Azure Web Role. (If you are using the ReportViewer control, we have enabled PDF generation for most applications.)
[1] https://social.msdn.microsoft.com/Forums/en-US/6ed5c738-390a...
>Components rely on GDI API may not work on Azure Websites. The workaround is moving to Azure Web Role. (If you are using the ReportViewer control, we have enabled PDF generation for most applications.)
[1] https://social.msdn.microsoft.com/Forums/en-US/6ed5c738-390a...
Is it a pipe dream to hope they go for DirectX in dotnetcore one day?
I'm so happy to read that they've gotten the image processing piece in place.
A while ago, on a bit of a lark and to see if I could, I started trying to port a copy of selenium webdriver to DNC and was initially surprised bow how successful I was in getting it to compile and build.
For the most part it was just re-pointing namespaces to their new locations.
When I got to the piece about image processing I hit a wall.
Maybe I'll take that on again sometime.
I'm so happy to read that they've gotten the image processing piece in place.
A while ago, on a bit of a lark and to see if I could, I started trying to port a copy of selenium webdriver to DNC and was initially surprised bow how successful I was in getting it to compile and build.
For the most part it was just re-pointing namespaces to their new locations.
When I got to the piece about image processing I hit a wall.
Maybe I'll take that on again sometime.
>Is it a pipe dream to hope they go for DirectX in dotnetcore one day?
What do you mean ? DX is Windows specific, .NET core is cross platform. And DX doesn't even have official .NET bindings on full .NET AFAIK.
OTOH I'm sure SharpDX and the likes could be ported (if it already isn't - this : https://github.com/sharpdx/SharpDX/issues/520 suggest it should work but I haven't tried it)
What do you mean ? DX is Windows specific, .NET core is cross platform. And DX doesn't even have official .NET bindings on full .NET AFAIK.
OTOH I'm sure SharpDX and the likes could be ported (if it already isn't - this : https://github.com/sharpdx/SharpDX/issues/520 suggest it should work but I haven't tried it)
Yes, SharpDX does work on .NET Core. Although it's not "official", it is essentially the de-facto library for DirectX bindings.
Offtopic:
I am the only one who absolutely hates charts like [0]these? Like what are those numbers on top, why it has no legend? Is it CPU utilization, miliseconds, seconds (maybe it was done over 1000 images or so)?
[0] - https://msdnshared.blob.core.windows.net/media/2017/01/Resiz...
I am the only one who absolutely hates charts like [0]these? Like what are those numbers on top, why it has no legend? Is it CPU utilization, miliseconds, seconds (maybe it was done over 1000 images or so)?
[0] - https://msdnshared.blob.core.windows.net/media/2017/01/Resiz...
In this case does it matter though? The purpose of the chart is relative performance, which it conveys minus a legend. Though I would agree that if they aren't including the legend, they should remove the axis and standard range labels entirely.
It matters at a quick glance - is performance measured in images per millisecond or milliseconds per image? The answer to that question is obviously important to how you interpret the relative performance. Both metrics (units per time and time per unit) are used widely in computing.
I was initially confused by the chart and yes the text (and following table) make the units clear, but the chart should be standalone.
I was initially confused by the chart and yes the text (and following table) make the units clear, but the chart should be standalone.
The legend is below each chart, with a table of the numeric results, with standard deviation. The code to reproduce the test is also provided, as well as the sample images.
They are milliseconds taken for the given task, as stated in the article.
Great to know, and good article. Can't help but feel that the author should have put in the full compressed images so we can view them. From the table at the bottom, Skia looks like it has quality issues, but it is kind of hard to say without seeing the full-sized compressed images. The uncompressed ones are on github, unless i've missed something, the compressed ones aren't.
What do you mean? The test is doing resize and JPEG compression. If you're interested in isolating the compression part, the code on GH should be super easy to modify to your particular requirements.
This seems like a really great use case for dotnetcore. I can see it being perfect for an image manipulation microservice.
Why wasn't Pixman/Cairo tested but Skia was tested? I'm sure it has to do with the natural availability of these things on .NET but I just checked and Cairo still has binding pre and post Mono. I would consider this option strongly there is way more support for Cairo in just about everything than Skia.
Because it's not Skia that is tested, but SkiaSharp, which is a .NET / Mono wrapper for Skia. It was included because the work to make it compatible with .NET Core is close to completion and because it's promising work. I hope this clarifies. If there were Pixman/Cairo .NET Core wrappers in existence or close to completion, I'd be happy to update the post with them.
Google's library destroys in benchmarks, but quality and filesize is lower which suggests to me it wasn't a fair test.
The quality settings should have been standardised to human-perceived image quality, given that's the purpose of an image.
The quality settings should have been standardised to human-perceived image quality, given that's the purpose of an image.
Unfortunately that's not possible. Each library makes choices and tradeoffs on their defaults. Each time, I set whatever quality dials existed to the highest available, and I standardized on 75 JPEG compression. There is no way to get consistent quality across all libraries, which is why I included the results for the 12 images used in the test in the blog post, so everyone can handle this the way they want.
There is also https://imageresizing.net/ with a new ImageFlow library for cross-platform
Yes, but that doesn't have a .NET Core library, and as such wasn't included in the post.
Does anyone know when .NET Core will get a decent linear algebra library?
I've heard MathNet.Numerics has a plan to add it, but so far there seems to not have been much progress.
I've heard MathNet.Numerics has a plan to add it, but so far there seems to not have been much progress.
5-10 megapixels input images might be more representative, as many phones produce such sizes.
It doesn't make a difference in the relative performance of the libraries on the metrics in the post, it only makes the benchmarks slower to run. This being said, the code is available, so this is easy for anyone to check. If I ever get to do a memory benchmark, that would probably be a different story, however.
If you need something right now, consider imagemagick or graphicsmagick.
In practice this meant that if you were cropping an image server-side in C# code on a Linux server, a C-rewrite of a Windows UI layer would kick in and do the work for you. Amazing work from the Mono team, because it worked, but also pretty nuts.
If there's one place I remember having stuff that "just worked" on Windows and had weird subtle quirks on Mono, it's image processing (I'm talking a few years ago). So IMO it's pretty awesome that they're replacing all that legacy with a decent 100% .NET image processing library.
(Sidenote: this was also the only real problem we faced developing a C# backend with a team on Windows, Linux and OSX computers, and running it all on Mono on Docker on Linux - all well before .NET Core. Mono really is/was that good)