HTML 5 Canvas Radial Gradients

Some of the trickiest bugs getting Azure canvas working were with radial gradients. Radial gradients in canvases are very flexible, more so than CSS radial gradients or as commonly found in paint software. Other than the spec,which is not particularly user-friendly, I did not find much explanation of how they work. So, here is my understanding.

A gradient is an interpolation between colours across a filled area. The simplest kind of gradient is linear, which is just a linear interpolation. For example, here is a rectangle with a left to right linear gradient from red to green:

A radial gradient interpolates circles, rather than lines, for example here is a simple radial gradient from red to green:

To add clarity, here is the same gradient with the start and end circles added:

The above example is simple because the two circles have the same centre and the 'start' circle is entirely within the 'finish' circle. In addition, there are only two colours involved, the start and end colours. These two issues are pretty much orthogonal, so we'll deal with multiple colours first, and we'll do so in the context of linear gradients, since the issues are the same independent of the type of gradient, and linear ones are simpler.

The start, end, and any intermediate colours are called colour stops. Each gradient must have at least two colour stops - the start and end colours - and may have any number of additional, intermediate colour stops. The syntax for these things is (I've elided the details of gradient creation for now):

  var context = canvas.getContext('2d');
  var gradient = context.create...Gradient(...);
  gradient.addColorStop(0, 'red');
  gradient.addColorStop(1, 'green');
  context.fillStyle = gradient;
  ...

The gradient will blend smoothly from red to green. The first parameter to addColorStop can be thought of as the parameter in a parametric equation; 0 is the start of the gradient, and 1 is the end. In fact, the gradient will shade from negative to positive infinity. Outside the start and end colour stops, the colour is extrapolated. For example, here is a gradient fill that shows the gradient outside the 0,1 range; the black lines are at 0 and 1 on the gradient:

Additional colour stops add intermediate colours, for example adding

  gradient.addColorStop(0.5, 'blue');

gives a gradient from red to blue to green (again the black lines are at the colour stops):

And here's a more complex example,

  gradient.addColorStop(0, 'red');
  gradient.addColorStop(0.1, 'blue');
  gradient.addColorStop(0.3, 'red');
  gradient.addColorStop(1, 'green');

in linear and radial versions:

OK, now back to just two colours, and more complex radial gradients. The syntax to create a radial gradient is:

  createRadialGradient(x0, y0, r0, x1, y1, r1)

where *0 define the position and radius of the starting circle (where the gradient parameter is 0), and the *1 arguments define the end circle (where  the gradient parameter is 1). If we offset the two circles (but circle 0 is still completely inside circle 1), then we get a skewed version of the simple radial gradient:

But, if the circles are not inside one another, then we get a cone:

Huh?

The gradient is interpolated from positive to negative infinity, just like with linear gradients. In the radial case, you can think of the gradient being drawn by stroking many incremental circles between the start and end circles. The position and radius of the circles are interpolated between x0, y0, r0 at t=0 (where t is the gradient parameters discussed above) and x1, y1, r1 at t=1. The interpolation is extrapolated to larger and larger circles as t tends to positive infinity, and to smaller and smaller circles as the radius disappears to 0, somewhere between t=0 and t=-infinity. This is what gives the cone shape. To illustrate, I have added the circles at t=0 and t=1 and a line along the centres of all the hypothetical circles:

The shading of the gradient is also kind of interesting; the shading is limited to the cone and is a gradient from the 'left' side of the t=0 circle to the 'left' side of the  t=1 circle. In particular, the gradient extends across the t=0 circle, but the t=1 circle is a solid colour. That is in contrast to the simple case (where t=0 circle is entirely inside t=1 circle) where t=0 is a solid colour and t=1 is shaded. To understand this we need two concepts: that the gradient is drawn from positive infinity to negative infinity (technically, towards negative infinity until the radius of circles becomes 0), and that what is drawn is not overdrawn. In the example, the gradient is drawn right-to-left, getting less green and more red as we go. The gradient goes between the left side of the circles because we draw from the right and do not overdraw. Note that left or right does not make a difference, we draw from circle 1 to circle 0, so the gradient will go from the side of circle 1 closest to circle 0, to the side of circle 0 furthest from circle 1.


There is nothing which says that circle 0 must be the smaller circle, so we can create a gradient from the 'opposite' sides of the circles by swapping the definition of the circles in the createRadialGradient call and the colours of colour stops:

The final case is if the two circles overlap, but neither is entirely contained in the other. We can apply the same understanding as the separate circles case (and of couse, that understanding applies to the simplest case too). In fact the gradient looks similar, just with a more compressed cone:

We can see the cone stretch out as the circles get further apart (in the first example, the circles are just one pixel apart):

 

 And there is the interesting case where the edges of the circles touch. This looks like the case where circle o is inside circle 1, except that the solid blue colour stops at the edge of the circles. One pixel further, and we are back to the simple case:

Orcon - why you so useless?

This is actually a rant about software design, but that won't become apparent until later...

I have been an Orcon (an internet/phone provider) customer for the last few years, I forget how many exactly, but somewhere between 1.5 and 3.5. As is par for the course for internet in NZ, the service is slow and expensive, but they have at least been good on the customer service front and they are relatively cheap (compared to other NZ providers). I've recently moved out of my house (not out of choice) and so had to terminate my contract early, there is a charge for this, which is fair enough, I had the option to choose a more expensive non-contract option when I signed up, but chose not to. It slightly irritates me that the shared house I'm now in is with Orcon too, so I'm paying to terminate my contract, but still with them. And it irritates me a lot that I can't pause my contract because I would probably have gone with them again when I move into my new house in a couple of months, not any more, obviously.

Anyway, the charge didn't turn up, and since Orcon emails have a habit of being spam filtered (see the title of the blog post), I thought I would give them a call. It turns out that " a request was made to cancel, but it didn't work". What? It didn't work?! How does a simple cancel request not work, and why does it fail silently? Why didn't you call or email? That is pretty dumb to start with. Of course the phone person didn't know why, but she promised to re-do the request, I suppose I will have to call back to check it works. This is not a good customer service protocol.

Then she asks if I have one of their modems, because if so, I will be charged if I haven't sent it back. So I have a modem, I have no idea if it is theirs, I suspect it is, but I acquired it several years ago and it has no Orcon stickers on it, so who knows? I do remember buying my own wireless router, so at worst I have a cheap wired-modem, probably worth $10 new. It does not seem making a fuss over, given I have been a customer from multiple houses for multiple years. But the really annoying thing is the customer service person has to ask me if it is theirs. Presumably someone at Orcon knows or they wouldn't be able to charge me for it. But the customer-facing people don't.

Here comes the software design bit - presumably this was an explicit constraint - they decided to deliberately separate the knowledge about who has their modems from the customer service record. WHY WOULD YOU DO THAT?! Who sits down and thinks that this is a good idea? What kind of idiot designs a system like this?

And breathe. So, can anyone recommend an internet/telecom provider in NZ?

More on the Layers refactoring

The refactoring is coming along nicely, I think the hard work is done - I have a set of abstractions I'm happy with, and most of the code has been refactored into the new architecture. There are a lot of rough edges still to iron out, and a lot of things marked TODO. There are also some of the Compositor related refactorings, which are not related to texture passing, still to do.

In this blog post, I want to outline the new abstractions for dealing with graphics and why I've chosen them, I'll mostly describe them as if they are new, but of course, Layers have been there all along, I'm mostly just breaking them up. The main abstractions are: shadowable layers, shadow layers, buffers, and textures. The latter two are divided up into host and client versions, where shadowable ~= client and shadow ~= host. The terminology should settle down as the refactoring finishes off.

There are different kinds of layers for different kinds of content, in particular image, canvas, and Thebes (these last are badly named and we should change the name to ContentLayers soon, thus Thebes ~= Content for Buffers, but more on that later). Colour layers don't transfer any kind of rendered graphics to the compositor, so don't matter here. Container layers are just for organising the layer tree, so also don't matter here. Shadowable layers are always drawn without HWA, so there is only the software backend. They should work with any compositing backend - Shadowable layers should be backend-independent.

A texture represents a lump of graphics data and is responsible for its memory and how that data is transmitted from drawing to compositing processes. Textures are backend dependent. Theoretically any texture can be used by any kind of layer, but in practice each kind of layer only uses a subset of kinds of texture.

A buffer abstracts how a layer uses its textures. A layer has a single buffer, and that buffer has one or more textures. In the simplest case, a buffer just has a single texture. Examples of more complex buffers are YUV images which have a texture for each channel, and tiled images which have a texture for each tile. The buffer defines how the textures are organised: when drawn, when transmitted to the compositor, and when (and how) they are composited. A layer only interacts with textures via a buffer, in fact layers do not even know that textures exist. Buffers are backend independent and (theoretically) can work with any kind of Layer and any kind of texture. In practice, each layer uses different buffers, and the names reflect this (CanvasClient, ImageClient, ContentClient are the different kinds of BufferClient). Also, the buffers are kind of picky about which textures they use. There is some re-use, for example canvas and image layers use the same kind of BufferHosts, and some kinds of texture are used by multiple kinds of buffer. Hopefully re-use will increase with time.

As an example of how this ties together, lets use a tiled Thebes layer as an example (in fact, I haven't converted tiled layers yet): some part of the layer is invalidated, the layer is responsible for knowing which parts are visible and valid, and therefore which regions must be updated. It tells its buffer to update this region, the buffer works out which textures must be repainted and repaints them (by calling back into the layer, because the repainting code is independent of buffers, textures, etc.). The buffer then tells the changed texture to update the compositor, which they do in their own way. On the shadow side, the invalidated textures are updated. When the screen needs updating, the shadow layer knows which region to draw, the buffer host knows how to map that region into textures and how to draw it to the screen.

Communication between renderer and compositor is on a per-layer basis, and I do not change this protocol. But instead of just sending a layer and some data, we now send a layer, a texture identifier, and some data. The texture client sends the message, a reference to the layer is passed in, and it knows its own identifier. On the compositing side, the message is received by the shadow layer (via ShadowLayersParent) and passed directly to its buffer host. The buffer host can then use the identifier as it chooses. The identifier includes the type of buffer and texture and this is verified. If the buffer has only one texture host, then it can be updated with the data from the message. If there is more than one, then the buffer host uses the texture identifier to work out which texture host should be updated. Buffer hosts/clients are free to use the identifier however they like.

Buffer and texture hosts and clients are not created directly, hosts are created by the compositor and clients are created by a specific factory (name TBC). A layer asks for a buffer client of a certain kind, the factory will create that buffer client if possible, or might return some other kind of client if not. Buffer hosts are created in a similar fashion when they are required (when a texture host is added, see later). Texture clients are usually created by a buffer client's constructor (but sometimes not, and sometimes later as well). Texture clients are also created with a call to the factory, the buffer client should know what kind of texture client it wants, and whether it can make do with some other kind if it is not available. When a texture client is created, a message is sent to the compositing side, and the compositor creates a corresponding texture host and adds it to the corresponding shadow layer, the layer should pass the texture host straight to its buffer host, creating one if necessary (what kind of buffer host is included with the message to create the texture host). Likewise destroying a texture client alerts the compositor to destroy the corresponding texture host. But, texture hosts can be created and destroyed at will, without alerting the drawing side. So, host/client pairs (buffers and textures) are loosely associated - there will 'always' be a pair, related by the combination of a layer reference and an identifier, but the exact objects in the pairing might change.

That's enough for one blog post, probably too much. Next time - types!

Azure canvas on by default

As of tonight, Azure canvas is on by default on all platforms! The last platform to get turned on was Linux, for which Anthony Jones did a lot of work on performance to bring Azure canvas up to par with Thebes, and exceed it in some cases. In the next cycle we will get to remove Thebes canvas from the code-base altogether.

The flavour of Azure canvas you get depends on the platform: Firefox OS, Android, and Linux get Cairo, Mac gets Quartz, and Windows gets Direct2D, if you have Vista or 7, and your drivers are up to date, and your graphics card is not blacklisted; otherwise you get Cairo.

You can see which Azure backend you are getting for Canvas (and possibly content) in about:support. You can also see the fallback backend, this is used if for some reason the preferred backend cannot be used, usually for very large canvases which are larger than the maximum texture size for Direct2D.

You can change the canvas settings in about:config: gfx.canvas.azure.enabled should be true, you can force Thebes canvas by setting it to false (for now). gfx.canvas.azure.backends contains an ordered list of backend names, for example, "direct2d, skia, cairo". Your preferred backend is the first backend in that list which your platform supports. Your fallback backend is the first backend on that list which your platform supports, is not the preferred backend, and is Cairo (which means you might not have a fallback backend).