How To Win Benchmarks And Influence People

ParticleAcceleration is a Microsoft demo. If you follow the link, you'll see lots of little spheres arranged in a big sphere, rotating slowly. It's kind of pretty. But as a benchmark, it is misleading. For many combinations of browser, platform, and graphics library, neither canvas nor math performance is the limiting factor; something strange is going on.
    
If you view the page source for the benchmark, then you'll see two background divs - the faint background gradient you can see behind the spinning balls. Why two? One is a gradient from #FFFFFF (white) to #BFBFBF (grey) and the other is from #FFFFFF to #BEBEBE (almost exactly the same grey). Every frame, the divs are swapped (to be precise: one is set to display:none and the other to display:block). This happens so fast and the gradients are so similar that the user cannot see the difference. I don't think there can be any purpose other than to confound browsers that are slow at repeatedly drawing gradients.
    
The demo presents itself as a benchmark of canvas performance*, not rapid switching between slightly different gradients. Releasing benchmarks like this is bad for the web. They put pressure on browser vendors (like us) to devote time and resources into optimising for artificial scenarios, and away from improving the real web experience for users.

Thanks to mayankleoboy1 for the initial bug report (806044) and Matt Woodrow for the detective work.  



*from the page: "Thanks for checking out this Internet Explorer 10 Platform Preview demo. This demo uses mathematical models to draw the particles in 3D, using the 2D Context HTML5 Canvas. The faster your underlying browser and computer, the higher your score and faster the particles will spin."

Appendix: some details, in case you are interested

The specific optimisation I imagine the benchmark is designed to exploit is caching gradients. Drawing gradients is expensive, so some caching makes sense. All modern browsers cache parts of a rendered web page. In Firefox, this can happen in multiple places, but primarily in the layers system. A gradient that doesn't change will only be drawn once into a layer and then retained. Once a rendered layer disappears (e.g., you move away from the web page or, as in this case, the div gets set to display:none) it is discarded (not straight away, but pretty soon). So for this benchmark we have to redraw the gradient when it reappears. On Windows, Direct2D caches gradients for us. So, if a gradient is redrawn soon after it was originally drawn it is cheap, and we are fast. I assume this is why IE is fast here too (Firefox, IE, and Chrome all get the maximum 60fps on my Windows machine).

Using the Cairo canvas backend (which we do for Linux, Android, older versions of Windows, and if your drivers are not up to date) knocks our frame rate from 60fps to 6fps. Here, we don't get the gradient caching behaviour from the graphics library. Removing the code from the benchmark that draws the gradients brings us back up to 11fps (yes, Direct2D is much faster than Cairo, even without the gradients. That is why we use it where we can!). So in the Cairo case, drawing the gradients uses about half our processor cycles. The Skia backend is even more affected by the gradients - 15fps and 50fps with and without the gradients. That is, the important parts of the benchmark use less than a third of the total cycles of the original benchmark.

It is possible to cache gradients within the browser. I think Webkit does this, and we are considering doing that within Azure. The downside to all that caching is that gradients take up a lot of memory and caching them in multiple places (and potentially in both CPU and GPU memory) is not good for our memory footprint. We have already had crashes due to running out of memory where a (faulty) driver has cached too many gradients for us.