Alpha Compositing(ciechanow.ski)
ciechanow.ski
Alpha Compositing
https://ciechanow.ski/alpha-compositing/
28 comments
> If we assumed a radially symmetric kernel, then coverage would just be some monotonic function of the distance from the pixel center to the edge
I don't think that's true. Consider a simple case where the shape is a square and you are sampling in such a way that the closest edge point is closer to a corner of the square than the filter width. Using a function of distance you will get the same result as when sampling far from the corner. But the coverage is different in the two cases.
I don't think that's true. Consider a simple case where the shape is a square and you are sampling in such a way that the closest edge point is closer to a corner of the square than the filter width. Using a function of distance you will get the same result as when sampling far from the corner. But the coverage is different in the two cases.
I’m just talking about rendering one edge (infinite line) separating the plane into two regions.
If you want to deal with pixels near multiple edges, it needs to be more complicated.
If you want to deal with pixels near multiple edges, it needs to be more complicated.
Better filters are much more expensive, because they need to incorporate the influence of so many more pixels. A simple Lanczos-3 which is far from perfect requires evaluating 7x7=49 input points for each output point.
I’m talking about figuring out the coverage of a vector shape, not about sampling some arbitrary rendered scene or transforming a pixel image (etc.).
Calculating the percentage of some radially symmetric kernel covered by one side of a straight line should be very cheap (it’s just a monotonic function of the closest distance from the pixel center to the edge).
If we want to extend that to shapes with multiple edges close to the pixel, or to curved edges, it will get harder, but I don’t think it’s necessarily inherently more expensive than the “calculate the coverage of a little square” problem.
Calculating the percentage of some radially symmetric kernel covered by one side of a straight line should be very cheap (it’s just a monotonic function of the closest distance from the pixel center to the edge).
If we want to extend that to shapes with multiple edges close to the pixel, or to curved edges, it will get harder, but I don’t think it’s necessarily inherently more expensive than the “calculate the coverage of a little square” problem.
The problem is that "figuring out the coverage" already assumes you're modeling the pixels as little squares, and that is not the optimal model for pixels.
You are misunderstanding me.
Imagine we have infinite samples (i.e. a continuous brightness function), and we are using some radially symmetric reconstruction filter. Then the value of each output pixel is the sum (2d integral) of the brightness at the sample points multiplied by our function evaluated at the distance to the sample.
If our continuous brightness data consists of one color on one side of a line and a different color on the other side of the line, then if we take the contribution of every point in the plane and integrate them, the output value at our pixel is going to be a combination of the two colors lerp(color1, color2, x(d)), where x is some monotonic function of the minimum distance between the pixel center and the line, which will depend on the specific reconstruction filter we are using.
But as for the little square model: the linked article proposed that because it is being used right now, today, in many shipping vector graphics rasterizers, e.g. for fonts.
I agree with you that it yields suboptimal results, so I am proposing an alternative.
Imagine we have infinite samples (i.e. a continuous brightness function), and we are using some radially symmetric reconstruction filter. Then the value of each output pixel is the sum (2d integral) of the brightness at the sample points multiplied by our function evaluated at the distance to the sample.
If our continuous brightness data consists of one color on one side of a line and a different color on the other side of the line, then if we take the contribution of every point in the plane and integrate them, the output value at our pixel is going to be a combination of the two colors lerp(color1, color2, x(d)), where x is some monotonic function of the minimum distance between the pixel center and the line, which will depend on the specific reconstruction filter we are using.
But as for the little square model: the linked article proposed that because it is being used right now, today, in many shipping vector graphics rasterizers, e.g. for fonts.
I agree with you that it yields suboptimal results, so I am proposing an alternative.
On that last note about alpha compositing I really wish there was a way to opt into correct alpha composting on the web. Colors deserve more love!
I imagined while reading the article that SVG must be implemented using these techniques. You imply it differs. How?
There is a "color-interpolation: linearRGB" option in SVG, which seems to be implemented by browsers. See http://tavmjong.free.fr/SVG/COLOR_INTERPOLATION/ for a more detailed examination.
Well, that page doesn't render correctly in Firefox, so I wouldn't say it is implemented by "browsers".
Sadly no mention of gamma correction, which is where many implementations of alpha blending fail.
It is discussed briefly at the end, but I agree with you that it should have been given more prominence.
This is a pretty thorough article. Why would you focus so much on gamma correction? How would you expand it?
If I were writing it, I might start with a physical interpretation of alpha as a blend of transmission and reflection of light (it basically does this) but emphasizing that has to be actual luminance, which RGB isn't. I'm not even sure I'd add a whole lot of content as rearrange it so the role of linearity in blending was closer to the top.
My hot take is that physical correctness for graphics is very much in vogue right now, but this is coming at the sacrifice of psychovisual and physiological properties of graphics.
Gamma-correct compositing is one of those things that is actually quite difficult to detect, by most viewers, in most situations. For that reason it does not deserve to be put at the top.
Gamma-correct compositing is one of those things that is actually quite difficult to detect, by most viewers, in most situations. For that reason it does not deserve to be put at the top.
Compositing may be hard to judge, but crappy gamma-incorrect zoom (same problem) is visible everywhere.
Did you ever notice how the brightness of a zoomed-out photographs suddenly changes? Look for images with fine high-contrast structures, like brightly lit branches or distant fences. For example https://en.wikipedia.org/wiki/Crown_(botany) - see how the picture gets brighter as it de-blurs? This is not intentional. More often a region is just a bit too dark.
Same with thin line art. Zoomed-out cross-hatching is the worst-case. When you do inking on a PC you may only work zoomed-out, so you can print later in much higher resolution. Once you look at the print, the emphasis is all wrong. Some lines disappear while others are much too strong. You are about to complain to the printer but zoom in to check and notice that the print was flawless.
Some older programs (e.g. feh) switch to nearest-neighbour interpolation while you are panning. This looks crappy but actually gives you the correct average brightness (because it doesn't do any color compositing). You can see the brightness jump back to incorrect values when panning stops.
Did you ever notice how the brightness of a zoomed-out photographs suddenly changes? Look for images with fine high-contrast structures, like brightly lit branches or distant fences. For example https://en.wikipedia.org/wiki/Crown_(botany) - see how the picture gets brighter as it de-blurs? This is not intentional. More often a region is just a bit too dark.
Same with thin line art. Zoomed-out cross-hatching is the worst-case. When you do inking on a PC you may only work zoomed-out, so you can print later in much higher resolution. Once you look at the print, the emphasis is all wrong. Some lines disappear while others are much too strong. You are about to complain to the printer but zoom in to check and notice that the print was flawless.
Some older programs (e.g. feh) switch to nearest-neighbour interpolation while you are panning. This looks crappy but actually gives you the correct average brightness (because it doesn't do any color compositing). You can see the brightness jump back to incorrect values when panning stops.
Because taking gamma correction into account when doing blend operations is pretty vital if you want good results.
To the contrary, non-gamma-correct compositing is very common, since so many programs (both historically and today) simply never bothered to do it correctly--but we get good results anyway. In most situations we wouldn't notice the difference, it usually takes a somewhat contrived example to illustrate it well. You can see that the example chosen blends between primary red and primary green, which is not a natural color combination. If you replaced one of the two colors with primary blue, even then the artifacts would be less apparent (because of blue's lower luminosity).
People currently shy away from making images where the current hacky method gives bad results (because nobody likes making ugly images). If implementations were fixed, they might not.
Beautifully written and illustrated article. The live examples are perfect. Back when I was in university this is what we imagined the future of technical literature would look like.
Really nice and thorough article.
One approach is to use four sample points which lets us represent four different levels of coverage: 0, 1⁄3, 2⁄3, and 1:
Four samples actually gives you five levels depending on how many points are covered: 0, 1, 2, 3, 4. So these correspond to quarters, not thirds.
Four samples actually gives you five levels depending on how many points are covered: 0, 1, 2, 3, 4. So these correspond to quarters, not thirds.
nice. already giving me some ideas for my silly little python-and-bird-video experiments (https://www.instagram.com/reductionist_earth_catalog/?hl=en). I was using the built-in compositing in pillow (PIL fork), but maybe I should try rolling my own alpha compositing module using numpy.
The original PIL was pure junk when it came to doing things correctly. Pillow is better but I haven't tried to quantify it yet.
[deleted]
I wish people would try to calculate the coverage of some other reconstruction filter instead of just separate little squares. I’m not convinced it would be more expensive, and would hopefully give better results (less griddy artifacts, better spatial resolution), especially for images that will be rotated or resampled later.
If we assumed a radially symmetric kernel, then coverage would just be some monotonic function of the distance from the pixel center to the edge, and this function could be approximated using some low-degree polynomial and computed very quickly.
(Maybe this has been done?)