Tuesday 8 July 2008
Using Arbitrary Elements As Paint Servers
The latest feature in my bling branch is the ability to use any element as a paint-server CSS background for another element.
There are a few motivations for this feature. Probably the biggest usage of the canvas "drawWindow" API extension in Mozilla is to create thumbnails of Web content. The problem is, drawWindow necessarily creates "snapshot" thumbnails. Wouldn't it be cooler if there was an easy way to create live thumbnails --- essentially, extra live viewports into Web content? Now there is. It should be pretty easy to add this to S5 to get slide thumbnails, for example.
Another feature that's popular these days is reflections. With element-as-background, plus a small dose of transforms and masks, reflections are easy. A while ago Hyatt introduced a feature in Webkit to use a <canvas> as a CSS background; that falls out as a special case of element-as-background.
So here's an example:
And here's the markup:
<!DOCTYPE HTML>
<html>
<head>
<style>
p { background:url(#d); }
</style>
</head>
<body style="background:yellow;">
<p style="width:60%; border:1px solid black; margin-left:100px;">
"Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud
exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute
irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia
deserunt mollit anim id est laborum."
</p>
<canvas id="d" width="50" height="50"></canvas>
<script>
var d = document.getElementById("d");
var iteration = 0;
function iterate() {
++iteration;
var ctx = d.getContext("2d");
ctx.save();
ctx.clearRect(0, 0, 50, 50);
ctx.translate(25,25);
ctx.rotate(Math.PI*iteration/180);
ctx.fillStyle = "lime";
ctx.fillRect(-10, -10, 20, 20);
ctx.restore();
setTimeout(iterate, 10);
}
iterate();
</script>
</body>
</html>
Unlike SVG paint servers, elements-as-backgrounds have an intrinsic size. Staying consistent with my earlier work on SVG effects for HTML, I define the intrinsic size as the bounding box of the border-boxes for the element.
As with SVG paint servers, an element-as-background is subject to all CSS background effects, including 'background-repeat' as in the example above.
Of course, the first thing any self-respecting computer scientist will think of when they read about this feature is, "can I exploit infinite recursion to create a hall-of-mirrors effect or sell exploits to gangsters?" No. We refuse to render an element-as-background if we're already in the middle of rendering the same element-as-background.
The builds I linked to in my previous post contain this feature. I've uploaded the above example and the reflection demo.
The next thing I have to do is to write up a spec proposal for all this work and get it discussed by the CSS and SVG working groups. Based on that feedback we'll figure out the best way to deliver this functionality in Gecko. Unfortunately, the approach of "make existing syntax applicable in more situations" is not amenable to using vendor prefixes to isolate experimental features.
Comments
Maybe display:none could just allowed on the element and not prevent it from rendering in other contexts.
display: paint-server
or something like that. :)
display:none would suck for us because we won't create a frame tree (i.e. renderobject tree) for such elements. In this implementation they won't work as background sources.
When we do external references, you'd be able to reference an element in an external document, which would be quite clean. We'd want an API to access the DOM of such documents, but that's something we're going to want anyway.
background:element(#a#b)
Where #a refers to element with ID 'a' in the styled document, which in this case is an element with a subdocument, and #b refers to element with ID 'b' inside that subdocument. This extension would also cover the issue that there is currently no way for a CSS stylesheet to reference an element in the styled document. However, this syntax would not let you reference an element in a subdocument of an external document ... sigh.
Anyway, I really like the flexibility you gain with this feature. And unlike the SVG stuff, this would work in regular HTML right now.
Very cool stuff. I can't believe some of this might get into 3.1.
Yes, especially if there was support for background-size.
I do like hyatts idea of a display: paint-server frame type that doesn't affect the rest of the frame tree, but that still allows the element to be used as a paint server. This could be useful to do a 'proper' implementation of svg:defs too.
Such a frame would probably behave like a block except maybe shrink-to-fit rather than taking up maximum width. And its size would be unaffected by the size of the page.
An alternative to explicitly having "display: paint-server", would be to say that the nearest "display:none" parent of a paint server is automatically given a paint-server frame. This avoids having to add explicit CSS syntax, and also covers the case where you have a paint server somewhere in the DOM which as part of the UI for the page is occasionally set to display:none.
None of this would be needed for the initial release of course.
The same is true for direct access to the canvas buffer (and you avoid the need for an additional element in your DOM).
You can also reuse the actual canvas image buffer when tiling, although I suppose you could optimize the paint server version to use canvas or image element image buffers as long as no scale is in effect and no border/padding/masks are present.
In practice I suspect slightly more direct access isn't going to matter much. The overhead is small. Doing the optimization you suggest would not be completely trivial since with element-as-background, styles applying to the canvas element are applied when rendering it as a background, so before we took direct access to the canvas buffer we'd have to check there were no borders, no background, no opacity, etc. Doable, but I wouldn't bother without a testcase showing it's a big win.
An alternative optimization would be to cache a surface containing the rendering of the element subtree. That would make multiple references to a complex subtree super-fast and would also take care of the canvas case (except use more memory). Actually in the past we've even talked about a generalization of that using heuristics to cache renderings of arbitrary subtrees to speed up multi-layer animations. Joost's Gecko extensions implement something like that, albeit with author control over the caching.
Slightly similar for this topic, I've wondered about adding support to cairo for "meshes" (for lack of a better word). It would basically be a Cairo meta surface with a recorded set of rendering commands that you can then use as a normal surface (and with it: source pattern). That would sound like a natural part to move caching to.
Ex: eAccelerator doesn't work with the SAPI you are using!