Blend2D – 2D Vector Graphics Engine(blend2d.com)
blend2d.com
Blend2D – 2D Vector Graphics Engine
https://blend2d.com/
42 comments
There are a few of those out there. Nanovg seems to be the one that fits your requirements most closely. If you're into Rust, lyon (plus something like gfx-rs) is another choice.
It's also worth noting that the HTML canvas implementations in most browsers is also not too bad for performance. Quartz is also really fast, but Apple made it really easy to accidentally use a slow rendering path (especially for bitmap operations)
It's also worth noting that the HTML canvas implementations in most browsers is also not too bad for performance. Quartz is also really fast, but Apple made it really easy to accidentally use a slow rendering path (especially for bitmap operations)
I have had great success using the ideas presented here (http://m1el.github.io/woscope-how/index.html) for simulation of traditional vector graphic screens. It is very fast and easily modified to allow for intensity modulation as well.
I really enjoyed your demo.
Thank you.
Just to clarify. This was not my work. I have just used the ideas presented therein for my own works and found them easy to adapt and improve upon.
Pathfinder fits the description: https://github.com/pcwalton/pathfinder
This AR demo on MagicLeap shows the potential: https://twitter.com/asajeffrey/status/1106667615622180864
This AR demo on MagicLeap shows the potential: https://twitter.com/asajeffrey/status/1106667615622180864
GPUs really just aren't designed for 2D vector graphics. There is a litany of approaches (three separate ones from pcwalton's Pathfinder R&D alone), and all of them have various issues and tradeoffs. If your GPU has enough power for one of the more advanced approaches like Slug or Pathfinder1's compute shaders, then your CPU likely has enough for Blend2D and the bandwidth for the resulting texture upload.
Pathfinder3 looks promising, but comes with its own tradeoffs. It's an open area of research.
Pathfinder3 looks promising, but comes with its own tradeoffs. It's an open area of research.
When listing approaches, people should look at Skia Compute[0]. While it is true that traditional GPU's are not a super-great match for 2D graphics, modern GPU's are becoming more and more CPU-like, just with more parallelism. I haven't done careful quantitative evaluation of compute myself, but don't feel that any understanding of modern 2D graphics performance would be complete without that.
[0]: https://github.com/google/skia/tree/master/src/compute/skc
[0]: https://github.com/google/skia/tree/master/src/compute/skc
Wow, I haven't seen this before, thank you for the pointer! This reminds me of some of the work in this 2014 paper which used a compute-based algorithm on the GPU to implement vector textures. And of course the works of GLyphy and PF1. http://w3.impa.br/~diego/projects/GanEtAl14/
If to speak about UI needs then CPU rasterization is just half of the problem.
320 PPI display (a.k.a. "Retina") has 9 times more pixels than old, classic 96 PPI one.
So just attaching new monitor will require 10 times better CPU in order to render the same UI. That if to use CPU rasterizers only.
Obviously that above is not an option. That's why Direct2D and Skia contain as GPU as CPU rasterizers. That dualism complicates things quite a lot - two alternative renderers under the same API roof shall produce similar results.
So Blend2D, to be a viable solution for the UI, shall be 10 faster in rasterizing than any current alternatives.
Yet, it was NV_path_rendering OpenGL extension from NVIDIA aimed for 2D path rasterization, but it seems the effort is dimmed now as OpenGL itself. OpenGL architecture, that was created to run H/W accelerated full screen apps, is far from being adequate for windowed UI.
So far Microsoft's Direct2D is the best thing that we have for H/W accelerated UI so far. And WARP mode in Direct2D (CPU rasterizer) is pretty close to the Blend2D - they also use JIT for rasterizing AFAIK.
320 PPI display (a.k.a. "Retina") has 9 times more pixels than old, classic 96 PPI one.
So just attaching new monitor will require 10 times better CPU in order to render the same UI. That if to use CPU rasterizers only.
Obviously that above is not an option. That's why Direct2D and Skia contain as GPU as CPU rasterizers. That dualism complicates things quite a lot - two alternative renderers under the same API roof shall produce similar results.
So Blend2D, to be a viable solution for the UI, shall be 10 faster in rasterizing than any current alternatives.
Yet, it was NV_path_rendering OpenGL extension from NVIDIA aimed for 2D path rasterization, but it seems the effort is dimmed now as OpenGL itself. OpenGL architecture, that was created to run H/W accelerated full screen apps, is far from being adequate for windowed UI.
So far Microsoft's Direct2D is the best thing that we have for H/W accelerated UI so far. And WARP mode in Direct2D (CPU rasterizer) is pretty close to the Blend2D - they also use JIT for rasterizing AFAIK.
It's true that increasing the size of framebuffer demands more from CPU as well. According to my experience a single core on a modern machine has no problem to render real-time into a FullHD framebuffer at high frame rate (depending on the content of course, but UI is fine). This means that multithreaded renderers using 4 threads should be able to render to 4K framebuffer without any issues. Since AMD will release 16c/32t consumer CPUs this year I see no problem on this front as we will have the computational power to run several multithreaded renderers at the same time.
Blend2D has multithreaded rendering on roadmap - I have experience in this topic and everything in Blend2D was designed with multithreading in mind (banding for example). The implementation I'm planning would scale very well.
NV_path_rendering - I haven't seen any detailed comparison to be honest. Frame-rate is not enough to compare CPU vs GPU technology - both memory consumption and power consumption are important as well to calculate frame-rate per watt.
I cannot comment on Direct2D as it's not open source and it runs only on a single operating system. So I don't consider Direct2D as a competition at the moment.
Blend2D has multithreaded rendering on roadmap - I have experience in this topic and everything in Blend2D was designed with multithreading in mind (banding for example). The implementation I'm planning would scale very well.
NV_path_rendering - I haven't seen any detailed comparison to be honest. Frame-rate is not enough to compare CPU vs GPU technology - both memory consumption and power consumption are important as well to calculate frame-rate per watt.
I cannot comment on Direct2D as it's not open source and it runs only on a single operating system. So I don't consider Direct2D as a competition at the moment.
Well, CPU rasterization is always O(N) complex (where N is number of pixels on screen). Multithreading here just adds constant multiplier that according to the math will still lead to O(N).
While GPU rasterization, from application perspective, is near O(1) - does not depend on number of pixels in ideal circumstances.
And having multiple threads to render UI is not desired - there are too many CPU consumers on modern desktop, e.g. online radio that is playing now, etc.
I am not saying that CPU rasterization makes no sense. Quite contrary. As a practical example: in Sciter on Linux/GTK I am using Cairo backend by default as OpenGL inside GTK windows is horrible. So Skia does not help there at all - Cairo and its CPU rasterizer is used.
If we would have something that allows to rasterize paths 5-10 times faster than current Cairo - it will solve all current desktop needs I think.
In principle 192 PPI resolution for desktop monitors of practical sizes (24 inch, 3840x2160 pixels) is OK - human eye will not be able to see separate pixels. Pretty much the same number of pixels is on mobiles ( iPadPro: 2732x2048 ). These are targets that need to be considered.
Practical requirements:
Take HN site in browser. Open it full screen. Decent 2D library should be able to rasterize that amount of text with 60 FPS (e.g. kinetic scrolling).
While GPU rasterization, from application perspective, is near O(1) - does not depend on number of pixels in ideal circumstances.
And having multiple threads to render UI is not desired - there are too many CPU consumers on modern desktop, e.g. online radio that is playing now, etc.
I am not saying that CPU rasterization makes no sense. Quite contrary. As a practical example: in Sciter on Linux/GTK I am using Cairo backend by default as OpenGL inside GTK windows is horrible. So Skia does not help there at all - Cairo and its CPU rasterizer is used.
If we would have something that allows to rasterize paths 5-10 times faster than current Cairo - it will solve all current desktop needs I think.
In principle 192 PPI resolution for desktop monitors of practical sizes (24 inch, 3840x2160 pixels) is OK - human eye will not be able to see separate pixels. Pretty much the same number of pixels is on mobiles ( iPadPro: 2732x2048 ). These are targets that need to be considered.
Practical requirements:
Take HN site in browser. Open it full screen. Decent 2D library should be able to rasterize that amount of text with 60 FPS (e.g. kinetic scrolling).
I don't know what rasterizers you refer to with:
I see no problem with multithreading, because it doesn't mean that all CPU cores will be busy with rendering, it means that the total time required to render a frame will be much lower while utilizing CPU power in a more distributed way instead of stressing a single core. You can use 2-4 threads on 8 core machine, for example, leaving the rest for other real-time tasks if required. Applications that use GPU for 2D rendering also use the full power of the GPU, if available.
Single core performance is stagnating while the number of CPU cores is increasing, so it's simply practical to design software to take advantage of that.
"CPU rasterization is always O(N) complex (where N is number of pixels on screen)"
I see no problem with multithreading, because it doesn't mean that all CPU cores will be busy with rendering, it means that the total time required to render a frame will be much lower while utilizing CPU power in a more distributed way instead of stressing a single core. You can use 2-4 threads on 8 core machine, for example, leaving the rest for other real-time tasks if required. Applications that use GPU for 2D rendering also use the full power of the GPU, if available.
Single core performance is stagnating while the number of CPU cores is increasing, so it's simply practical to design software to take advantage of that.
This is such a well-made landing page. Tells me what it does, what makes it good, isn't full of vague marketing waffle, doesn't put one sentence per screen, and has actual code and output examples!
This is very interesting. Especially since other vector engines are either unmaintained (like Antigrain) or extremely hard to build (Skia, Cairo).
I do wonder if these kind of libraries shouldn't be supplanted by hardware accelerated rendering, though.
I do wonder if these kind of libraries shouldn't be supplanted by hardware accelerated rendering, though.
> are either unmaintained (like Antigrain)
The author of Antigrain passed away a few years back, sadly.
It was and still probably is one of the best graphics libraries out there, and he was really hoping for a time when antialias-free, vector-based UI on high-pixel-density screens would become reality.
[1] Announcement in Russian: https://rsdn.org/forum/life/5377743.flat
"Maxim Shemanarev 1966-2013. Tragically, unexpectedly passed away in his home at 47." The discussion that follows names epileptic seizure as the cause of death.
[2] For example, this glorious text on text rasterization from 2007 http://www.antigrain.com/research/font_rasterization/index.h...
The author of Antigrain passed away a few years back, sadly.
It was and still probably is one of the best graphics libraries out there, and he was really hoping for a time when antialias-free, vector-based UI on high-pixel-density screens would become reality.
[1] Announcement in Russian: https://rsdn.org/forum/life/5377743.flat
"Maxim Shemanarev 1966-2013. Tragically, unexpectedly passed away in his home at 47." The discussion that follows names epileptic seizure as the cause of death.
[2] For example, this glorious text on text rasterization from 2007 http://www.antigrain.com/research/font_rasterization/index.h...
Skia is actually not too hard to build once you're used to gn and ninja. Download both of those tools and add them to your PATH, then cd into Skia's repo (that you just cloned):
$ python tools/git-sync-deps
$ gn gen out/release -args="is_official_build=true is_component_build=false"
$ ninja -C out/release
$ gn args out/release --list --short
$ gn args out/release
$ ninja -C out/release
include
include/config
include/core
include/effects
include/gpu
Skia is not so terrible, you just have to use their slightly obscure build tool. And it produces >8GB build artefacts with ~50 or more dependencies... Come to think of it is a bit of a nightmare.
I tried to use it a few times, but having to configure a Google build environment, without Google class workstations, meant I always gave up in the process and used Skia.Sharp instead.
I explained how to build it here, give it a try: https://news.ycombinator.com/item?id=19586159
Thanks for the heads up, but doing it across Windows/UWP, Android and iOS requires a bit more of effort than what you described, hence Skia.Sharp instead.
I'm doing exactly that though. To compile for iOS for example, create a separate output directory with `target_os="ios" target_cpu="arm64"`. You can see a few more examples here https://skia.org/user/build
Unfortunately, C++ libraries are a bit harder to use than other languages, hopefully the modules proposal will make this a lot easier.
Unfortunately, C++ libraries are a bit harder to use than other languages, hopefully the modules proposal will make this a lot easier.
Just in case: Skia can be built without Google build environment. For the Sciter (https://sciter.com) I just included all needed files into projects ( MSVC and XCode ). That setup takes some initial time but after that it works as any other library like libpng, etc.
This is what we really need, something that outputs an API agnostic set of render commands, (shaders, uniforms, mesh layouts etc) which auto clip themselves and such, so they can just be integrated across various engines and renderers.
Of course a better format in the first place(mentioned in another comment) would just solve this anyway
Of course a better format in the first place(mentioned in another comment) would just solve this anyway
This is similar to the approach libraries like ImGui take. They abstract the rendering backend exactly in this way.
How does the performance of path filling compare to font-rs?
Blend2D uses dense cell-buffer similarly to font-rs, however, it works quite differently and this difference allows Blend2D to be efficient even when rendering large paths:
- Dense cell buffer, 32-bit integer per one cell (FreeType/Qt use sparse cell buffer and two 32-bit integers per cell, font-rs uses float if I'm not mistaken)
- Blend2D builds edges before rasterization step, edge builder is optimized and performs clipping and curve flattening
- Rasterization and composition happens in bands, thus storage required by the cell buffer is quite small (currently band is 32 scanlines, but we will make it adjustable based on width)
- To complement dense cell buffer Blend2D uses also a shadow bit-buffer to mark cells that were altered by the rasterizer
- Compositor scans bit-buffer instead of cell-buffer to quickly skip areas that were not touched by the rasterizer
- Compositor is SIMD optimized and calculates multiple masks at the same time (at the moment it works at 4 pixels at a time, but this can be extended easily to 8 and 16 pixels)
- Compositor clears both cell-buffer and shadow bit-buffer during composition so when the compositor finishes these buffers are zeroed and ready for another band
- Blend2D maintains small zeroed memory pool that is used to quickly allocate cell and bit buffers when you create the rendering context and returned to the pool when you destroy it
There are probably more differences, like parametrization of NonZero and EvenOdd fill rules, etc, but these are really implementation details to minimize the number of pipelines generated by common rendering operations.
The advantage of font-rs is rendering small paths, large paths will have increasing overhead as a lot of computations would happen on zero cells. Blend2D rasterizer is universal and was tuned to perform well from small to 4K framebuffers.
When I was designing Blend2D's rasterizer I wrote around 20 rasterizers and benchmarked them against each other in various resolutions. I had similar rasterizers like font-rs (but not using floats) and these were only competing in very small resolutions like 8x8 and 16x16 pixels. When shifting to larger resolutions these rasterizers would always lose as shadow bit-buffer scan is much quicker than going through zero cells, especially if you do pixel compositing.
There are demo samples in bl-samples-qt repository that render animated content and have Blend2D and Qt rendering options. You can check them out to see how the rasterizer competes against Qt.
Let me know if that answers your question.
- Dense cell buffer, 32-bit integer per one cell (FreeType/Qt use sparse cell buffer and two 32-bit integers per cell, font-rs uses float if I'm not mistaken)
- Blend2D builds edges before rasterization step, edge builder is optimized and performs clipping and curve flattening
- Rasterization and composition happens in bands, thus storage required by the cell buffer is quite small (currently band is 32 scanlines, but we will make it adjustable based on width)
- To complement dense cell buffer Blend2D uses also a shadow bit-buffer to mark cells that were altered by the rasterizer
- Compositor scans bit-buffer instead of cell-buffer to quickly skip areas that were not touched by the rasterizer
- Compositor is SIMD optimized and calculates multiple masks at the same time (at the moment it works at 4 pixels at a time, but this can be extended easily to 8 and 16 pixels)
- Compositor clears both cell-buffer and shadow bit-buffer during composition so when the compositor finishes these buffers are zeroed and ready for another band
- Blend2D maintains small zeroed memory pool that is used to quickly allocate cell and bit buffers when you create the rendering context and returned to the pool when you destroy it
There are probably more differences, like parametrization of NonZero and EvenOdd fill rules, etc, but these are really implementation details to minimize the number of pipelines generated by common rendering operations.
The advantage of font-rs is rendering small paths, large paths will have increasing overhead as a lot of computations would happen on zero cells. Blend2D rasterizer is universal and was tuned to perform well from small to 4K framebuffers.
When I was designing Blend2D's rasterizer I wrote around 20 rasterizers and benchmarked them against each other in various resolutions. I had similar rasterizers like font-rs (but not using floats) and these were only competing in very small resolutions like 8x8 and 16x16 pixels. When shifting to larger resolutions these rasterizers would always lose as shadow bit-buffer scan is much quicker than going through zero cells, especially if you do pixel compositing.
There are demo samples in bl-samples-qt repository that render animated content and have Blend2D and Qt rendering options. You can check them out to see how the rasterizer competes against Qt.
Let me know if that answers your question.
It does, thanks! This does indeed look like good stuff. It is true that font-rs is not fully optimized for large areas, as it depends on memory fill and cumulative sum for each pixel. These are not hugely expensive, but it sounds like a sophisticated approach (as taken by Blend2D) can do better. Happy to see the work.
[deleted]
Whenever SVG rendering engines come up, sooner or later the question of "which part of the SVG spec does it render incorrectly?" comes up. How does this one fare?
I get the impression that often these rendering bugs are partially caused by old optimizations that break later spec changes or missed some edge cases the first time around, so I hope that means this engine does better than most because it's more recent, built from-the-ground-up and can learn from the mistakes of others.
I get the impression that often these rendering bugs are partially caused by old optimizations that break later spec changes or missed some edge cases the first time around, so I hope that means this engine does better than most because it's more recent, built from-the-ground-up and can learn from the mistakes of others.
Blend2D isn't a SVG rendering engine; maybe a SVG rendering engine can be built on top of Blend2D. But anyway SVG specifications are a hell (and these include a lot of features that are not related to the pure rendering). How do i know? I'm one of the developers of https://www.amanithsvg.com
Things like this convince me more and more that SVG is overdesigned and solving the wrong problem:
The world needed a vector version of PNG far more than it needed a vector version of HTML/CSS.
The world needed a vector version of PNG far more than it needed a vector version of HTML/CSS.
W3C is creating a subset of SVG called "SVG Native". It is intended for icons and simple graphics, so it won't have support for text, scripts, links or XSL processing. Also, use of CSS or animations isn't recommended. It will be similar to SVG in OpenType fonts: https://docs.microsoft.com/en-us/typography/opentype/spec/sv...
Neat. Hopefully it gets adopted. And hopefully it uses its own file-extension so I can actually tell them apart - I'd love to have, like ".svgn" for native svg.
Another subset to join the ones already being used by UWP, Android and iOS. Oh well, xkcd comes to mind.
Wasn't there already "SVG Tiny" for this purpose? I recall it being used on mobile a long time ago, like circa 2005.
I will push back a bit and say SVG's integration with HTML documents is very convenient and useful. I think it has been held back by poor implementation, particularly Mozilla's.
The SMIL stuff has made the SVG DOM a bloody mess, and it's deprecated everywhere, which is a sad outcome, but it's still useful.
The SMIL stuff has made the SVG DOM a bloody mess, and it's deprecated everywhere, which is a sad outcome, but it's still useful.
That makes no sense. SVG _is_ the vector equivalent of PNG. Just because it's defined in an xml namespace that can be directly embedded in html, doesn't mean it is somehow bound to it.
I can't write JavaScript inside of a PNG. I can't reference external resources inside of a PNG. I can't reference CSS classes inside of a PNG. I can't animate a PNG. A PNG is a simple, dumb, uncomplicated file with a minimal surface area for vulnerabilities.
Definitively, this.
I would love to find some kind of stripped-down vector graphics engine that is fully geared towards real-time rendering of path-based graphics for games. For example to write retro-style games that use 2D vector graphics instead of textured quads. I've been writing a simple game that uses this graphics style, and so far I've been simply using pre-rendered textures for sprites. Being able to directly render these things by composing vector-based layers would allow all kinds of interesting effects and image quality improvements, e.g. seamless scaling morphing, etc. Obviously this would be way, way slower than emulating these things using bitmap textures and/or SDF's for scaling, but for the simple graphics my game is using that should probably be fine, even on mid-range hardware there is plenty of frame time I could dedicate to vector rendering and still hit at least 30fps.