Azure Canvas

Other than the mask layers stuff, most of my work has been on getting Azure canvas working, passing tests, and landed in the tree. Most of the work has been done before, my contribution has been on debugging, testing, and wiring things together, and of course a few interesting edge cases. To describe the work, I'll give a little background on our graphics libraries.

Back in the day, 'all' rendering in Firefox was done using the Cairo graphics library. This was wrapped in a thin wrapper library called Thebes (Cairo is the external library which does all the work, Thebes is the internal wrapper). We would like to support different graphics libraries and we would like a more efficient wrapper layer, and so was born Azure. This is a new internal system which abstracts a lot of our drawing code. Multiple backends can be slotted into Azure, and, in turn, Azure can be used as a backend for Thebes instead of using Cairo. So, for example, our graphics stack could use Thebes on top of Azure on top of Cairo (or Skia or Direct2D, etc.). Azure has some interesting advantages over Thebes, mainly that it is a state-less API (well, except for a transform and clipping rect), Bas and roc have written some interesting blog posts about this before.

The long term plan is for Thebes to be removed, and Azure to be used directly. For now there are people implementing content rendering using Thebes on top of Azure (on top of Direct2D on Windows, Core Graphics on Mac, and Skia or Cairo elsewhere).

HTML 5 canvas uses a different rendering path from the rest of content. We actually have two canvas implementations right now, one on top of Thebes and one on top of Azure. Currently the latter is used only on Windows where Direct2D is present, and the Thebes canvas is used elsewhere. We want to remove the Thebes canvas, because having two canvas implementations is a maintenance burden and affects our DOM bindings implementation.

So, I have been working on getting Azure canvas working with both Skia and Cairo backends, the former only on Windows, the latter everywhere, with Anthony Jones. It is all about ready to land, after that, we'll let it settle for a while to ensure there are no problems, and then we can remove Thebes canvas completely. Some of the interesting bits of this work have been using Skia canvases as source for tab previews on Windows, organising different backend preferences on different platforms, and ensuring we can always fall back to a safer backend, and just getting all the complex logic in Azure canvas and the various backends right. Plus the occasional memory leak, and some interesting interactions between the layers system and Azure.

In terms of the bigger picture, this work should lead to Azure totally replacing Thebes for canvas rendering. Elsewhere, Azure is being developed into the primary backend for Thebes, so that it will be used everywhere for rendering content, rather than using Cairo directly. Then we will work to remove the Thebes layer altogether so we are rendering directly with Azure, this will be fiddly because there is Thebes code everywhere in the code base. But at the end of it all, we should have faster, more efficient code, and be able to easily plugin different graphics backends, which is a win.

Mask layers on the basic backend

Using mask layers on the basic backend is a bit different from the GPU backends because we don't use shaders, and instead must do a bit of a dance with Cairo. Also, the transforms are different in basic layers vs. the GPU layers (relative to user space, rather than relative to the layer's container), which means we must handle the layer transform differently.

When using basic layers, the drawing is done using Cairo, or more precisely, using Thebes our wrapper graphics library. Therefore, we do not (as on the GPU backends) use shaders to do the masking, but resort to actual mask operations. The basic idea is we draw the content un-masked, use that as a source, use the mask layer as the mask, and just paint the source through the mask. Of course it gets more complicated, because sometimes the source layers use paint, and sometimes fill. Furthermore, if the source layer has an opacity, then we must do a push/pop group because there is no mask with opacity operation in Cairo. In this case, we push the current drawing context, do the paint or fill, then use this new context for the masking source.

Thebes layers are more complex because they use a rotating buffer for drawing to minimise redrawing. Masking is not affected too much though, for each quadrant we just mask as if we were drawing the whole layer, using the mask's transform and ignoring any rotation. The only slight complication is that the mask and its transform must be extracted from the source layer in the basic layers code, and passed to the buffer separately.

The transforms in basic layers work differently to the other backends. In layout, we set up the mask transform the same way, but we use a different method of calculating the effective transform. The actual computation for mask layer transforms is the same, but the matrix passed to ComputeEffectiveTransformForMaskLayer will be different. When we come to rendering, we use only the mask layer's transform for when using the mask, that is, we use SetMatrix, we don't add the mask's transform to the current transform like we do on the GPU backends.

The basic layers backend is always used on the drawing thread when we use OMTC (either the OpenGL or basic backends will be used for compositing). Masking is mostly done on the compositing thread. IPDL does most of the heavy lifting for this, the only thing we have to do is to ensure that if we are only drawing (that is, not also compositing), then we must 'draw' the mask to an off-screen buffer so that it can be used by the compositing thread. Masking is only done if we are compositing also, so when we draw a layer, we check whether to draw it with the mask (if we are compositing) or draw it and its mask separately (if we are only drawing). Of course there is an exception, colour layers are masked on the drawing thread.