Intro

In #985 and #989 @panxinmiao makes a case for a simpler blending mechanic. I was very hesitant at first, but when I realized dithered blending can still fit in, I fell for its simplicity, and the ability for user to exert more control over how objects are blended.

This means we'd say goodbye to the weighted blend modes and the ordered2 blend mode.

I don't mind dropping the "weighted", "weighted_depth", and "weighted_plus" blend modes; they were not as useful as I hoped. I doubt anyone uses them. For those who do, sorry! Actually weighted can still be supported. The weighted_plus will likely be dropped, but I plan to create an example to replicate it.

The "ordered2" blending, however, helped make things more fool-proof. The purpose of this post is to list the advantages that we'd lose, the potential issues that our users run into, and how to deal with that.

Sorting by depth

In some cases, sorting the objects by depth ensures correct blending. People who create 3D scenes generally design them such that this is the case. But if the data is given, you have to deal with it, and ordered2 really helped rendering such scenes. A simple example is a point cloud (gfx.Points).

What does ordered2 do?

This blend mode uses two "passes", one for opaque objects, and one for transparent objects. The trick is that objects that have both opaque and transparent fragments get rendered in both passes. That way, the opaque fragments write depth, while the transparent fragments do not.

This means that 1) users don't have to think about their objects being transparent or opaque, because they automatically get handled appropriately based on the alpha produced by the shader; and 2) you can have objects that have both opaque and transparent parts, and all is well. Case (2) is plentiful in Pygfx because of the aa edges in lines, points, text, grid, ...

It's not a silver bullet though; correct blending still relies on the order, and the convenience is at the cost of performance. But objects are not hidden, and artifacts are less likely to occur.

The theory

Consider a case where an object consisting of both opaque and transparent regions is drawn (the rectangular parts below). They write to the depth buffer. And now new fragments (either opaque of transparent) are drawn in three locations (the circles):

Implications

For objects that have relatively large transparent surfaces, this sucks, because it hides fragments that are behind the transparent surfaces that were drawn earlier.

For aa edges, a use-case which is more prevalent, it means that the edges hide the stuff behind it. This does not sound too bad, but consider a white backgound, so the aa edge fades to white, when a darker object is drawn behind, you'd get a white artifact which looks weird, also see the example in the next section.

A demo case

The point cloud in the fly_controller.py example is a nice demo. With ordered2 it looks like this:

Without ordered2, this is the first result I got, because depth_write is enabled:

Forcing depth_write=False fixes it.

Using the marker material and using opaque colors triggers the aa edges case. You can see how the black edge was once blended with a green marked behind it, but now a blue marker was drawn behind it, causing the green artifact.

You don't see such artifacts with ordered2.

What we can do in Pygfx

• Make sure the sorting works well out of the box: background and similar objects should have a lower default render_order.
• Some form of detecting whether an object should be transparent would be nice.
• Same for depth_write.
• Make it clear and simple how to mark an object as transparent.
• Examples and docs.
• Implement FXAA and other post processing filters, so aa for smooth edges is not necessary, and dithered blending looks less noisy.

Options to mitigate user issues

Artifacts in aa edges

• Turn of aa 🤷
• Make the object transparent or set depth_write to False.
• In some cases increasing therender_order of the object can help.

Blending weirdness

• Use material.blending = 'dither'. It looks noisy but it's perfect!
• In some cases increasing therender_order of the object can help.
• Make the object transparent or set depth_write to False.
• Make some objects opaque to avoid blending. Maybe it's not actually needed.

Objects with both opaque and transparent parts

• Use material.blending = 'dither'.
• Split the object up in an opaque part and a transparent part, if possible.
• The above can be done with a trick that replicates the behavior of ordered2 blending: create two versions of the object, one with transparent=False, alpha_test=0.999, and one with transparent=True, alpha_test=-0.999.