Wednesday 4 June 2008
Applying SVG Effects To HTML Content
Introduction
One of the problems with the standards-based Web is that it's hard to use SVG's features to enhance HTML content. For example, there is no reasonable way to clip an HTML element to a non-rectangular region, or to apply an alpha mask to an HTML element, or to apply image processing effects such as color channel manipulation to HTML elements. SVG has these features but they can only be applied to SVG elements. You can embed the HTML in a <foreignObject>, but that has major limitations; in particular, you can't apply effects to HTML elements via stylesheets without changing the markup --- ripping out content and putting it inside ugly SVG fragments --- and breaking most CSS layouts in the process.
One approach to solving this is to create new CSS properties for the desired effects. Dave Hyatt's been doing this a bit in Webkit, and the approach has its place. However for complex effects such as clipping to arbitrary shapes and custom image processing, CSS isn't really up to the task. One problem is that CSS isn't good at manipulating structured values like shapes and filter processing stacks; they're cumbersome to write in CSS expression syntax, or else they require new custom CSS syntax (e.g. @-rules), and there's no standard DOM to let scripts manipulate components of these structured values. Another issue is that we should
try to avoid duplicating specification and implementation of complex features.
So I've been experimenting with better ways to apply SVG effects to HTML content. The first step is to make SVG's 'clip-path',
'mask' and
'filter' properties work when applied to HTML content.
clip-path
Here's some XHTML markup that clips elements to a shape composed of a circle next to
a rectangle.
<html xmlns="http://www.w3.org/1999/xhtml"
xmlns:svg="http://www.w3.org/2000/svg">
<body style="background:#ccc; font-size:30px;">
<style>
p { width:300px; border:1px solid black; display:inline-block; margin:1em; }
iframe { width:300px; height:300px; border:none; }
b { outline:1px dotted blue; }
</style>
<p class="target" style="background:lime;">
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt
ut labore et dolore magna aliqua. Ut enim ad minim veniam.</p>
<iframe class="target" src="http://mozilla.org"/>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing
<b class="target">elit, sed do eiusmod tempor incididunt
ut labore et dolore magna aliqua.</b> Ut enim ad minim veniam.</p>
<style>.target { clip-path: url(#c1); }</style>
<svg:svg height="0">
<svg:clipPath id="c1" clipPathUnits="objectBoundingBox">
<svg:circle cx="0.25" cy="0.25" r="0.25" id="circle"/>
<svg:rect x="0.5" y="0.2" width="0.5" height="0.8"/>
</svg:clipPath>
</svg:svg>
</body>
</html>
So we have a block element, an <iframe>, and an inline element all clipped by the same clipPath. The clipPath shape coordinates are relative to the bounding-box of the clipped object.
Here's the rendering:
You can't see it in this example, but hit-testing is affected by clip-path as you'd expected; mouse events in the clipped-out area pass through to the element(s) underneath.
One of the tricky integration points is handling elements which generate multiple CSS boxes, such as the inline element here. I don't expect clipping and masking to find much use there, but filters can be useful for inlines. The approach that makes most sense is to apply the effects to the whole element at once, and make the SVG "object bounding box" be the rectangle union of all border-boxes for the element and its geometric descendants, plus any outlines on the element or descendants. This is useful and also seems in accord with the spirit of the SVG spec.
clipPath, mask and filter content can also use "userSpaceOnUse" units. In SVG,
"user space" is established by the SVG viewport containing the affected content. We don't have such a viewport for non-SVG content, so I make "user space" be the rectangle which is the union of all border-boxes for the affected element. User space is in CSS pixel units, so clipPath, mask and filter content can specify lengths in CSS pixels as well as percentages relative to the size of the affected element.
Scripting
It's easy to add some dynamism to the above example:
<button onclick="toggleRadius()">Toggle radius</button><br/>
<script>
function toggleRadius() {
var circle = document.getElementById("circle");
circle.r.baseVal.value = 0.40 - circle.r.baseVal.value;
}
</script>
The SVG DOM isn't as clean as it could be, but it's adequate.
mask
Replace the 'clip-path' chunk above with 'mask':
<style>.target { mask: url(#m1); }</style>
<svg:svg height="0">
<svg:mask id="m1" maskUnits="objectBoundingBox" maskContentUnits="objectBoundingBox">
<svg:linearGradient id="g" gradientUnits="objectBoundingBox" x2="0" y2="1">
<svg:stop stop-color="white" offset="0"/>
<svg:stop stop-color="white" stop-opacity="0" offset="1"/>
</svg:linearGradient>
<svg:circle cx="0.25" cy="0.25" r="0.25" id="circle" fill="white"/>
<svg:rect x="0.5" y="0.2" width="0.5" height="0.8" fill="url(#g)"/>
</svg:mask>
</svg:svg>Now the rectangle is being painted with a translucent gradient and the shape is being used
as an alpha mask instead of just clipping.
Here's the rendering:
As per the SVG spec, masks do not affect hit-testing, so events will be received even where content is masked out. It would be smart to implement the SVG
pointer-events property for non-HTML content to give authors a way to control this.
filter
Replace the 'mask' chunk with some filters:
<style>
p.target { filter:url(#f3); }
body:hover p.target { filter:url(#f5); }
b.target { filter:url(#f1); }
body:hover b.target { filter:url(#f4); }
iframe.target { filter:url(#f2); }
body:hover iframe.target { filter:url(#f3); }
</style>
<svg:svg height="0">
<svg:filter id="f1">
<svg:feGaussianBlur stdDeviation="3"/>
</svg:filter>
<svg:filter id="f2">
<svg:feColorMatrix values="0.3333 0.3333 0.3333 0 0
0.3333 0.3333 0.3333 0 0
0.3333 0.3333 0.3333 0 0
0 0 0 1 0"/>
</svg:filter>
<svg:filter id="f3">
<svg:feConvolveMatrix filterRes="100 100" style="color-interpolation-filters:sRGB"
order="3" kernelMatrix="0 -1 0 -1 4 -1 0 -1 0" preserveAlpha="true"/>
</svg:filter>
<svg:filter id="f4">
<svg:feSpecularLighting surfaceScale="5" specularConstant="1"
specularExponent="10" lighting-color="white">
<svg:fePointLight x="-5000" y="-10000" z="20000"/>
</svg:feSpecularLighting>
</svg:filter>
<svg:filter id="f5">
<svg:feColorMatrix values="1 0 0 0 0
0 1 0 0 0
0 0 1 0 0
0 1 0 0 0" style="color-interpolation-filters:sRGB"/>
</svg:filter>
</svg:svg>
Filters are cool so I decided to go a bit over the top here. The block element gets an edge-detection convolution filter. The <iframe> gets converted to grayscale.
The inline element is blurred. When you hover over the body, the filters change; the <iframe> gets the edge detection filter, the block gets a color channel transformation that takes the alpha channel to the green channel, creating a punch-out effect, and the inline element gets a 3D lighting effect.
Here are the renderings:
Filters do not affect hit-testing.
Discussion
All these effects are fully live. You can select, zoom, change the DOM, invoke contenteditable, etc, and everything works fine. Effects are applied to the drawing of the selection, and even to the drawing of the caret while it's in an affected element.
The effects following the SVG composition model; if an element has a filter, clipping and masking, then the filter is applied first followed by the clipping and/or masking. They behave like CSS 'opacity' in that they induce a pseudo-stacking-context for the element. They do not affect layout, although 'filter' can produce drawing outside the bounds of the element (e.g., shadows). Currently, in my implementation, 'filter' can actually affect layout because when 'filter' paints outside the element we may show scrollbars so that the user can scroll to see the overflowing paint, but I think this should be considered a bug.
Since we don't yet support remote content references in Gecko, you have to put the effects fragments inline in your document. This is ugly and also means you can't use these effects in non-XML HTML. Once we implement remote content references, these problems will go away; author CSS style sheets will be able to reference effects in auxiliary SVG documents. Many effects can be stored in a single SVG document, amortizing the cost of an extra resource. Authors will even be able to use tools like
Inkscape to generate SVG effects and reference them from CSS.
At a spec level, there's very little that needs to be said. No new syntax is required.
A specification needs to be written documenting decisions on the issues I've mentioned above (not necessarily the same decisions I made :-) ).
In lieu of a spec, we have to decide how much of this to take in Gecko and when. It's nice that there's no new syntax, but that also means there's no convenient way to use -moz prefixing to isolate the non-standard features. We could create new
'-moz-filter'/'-moz-clip-path'/'-moz-mask' properties that behave like the existing properties except they also apply to HTML. Maybe it's not worth it. Something for discussion.
I'm making tryserver builds right now, and I'll update this post with a link when they're ready. Here's a link to my Mercurial repository. (Unfortunately, I accidentally pushed copies of NSPR and NSS to that repo, so don't blindly pull from it unless you're OK with that.) Some demos:
At some point I'll post a followup talking about the other work I've done on this branch that leads up to these features, with some more details about the implementation. I'm also planning to add some more features soon.
Aside
A nice side effect of providing better SVG-HTML integration is that it gives SVG a leg up on the Web. You can't do these effects using Flash or Silverlight, and since they're not standards they probably won't ever be invited to this party.
Comments
John: not sure. It's at the intersection of the CSS and SVG working groups, probably should be a combined thing.
I think you have just started a revolution. Finally HTML is not like a newspaper or magazine. It is what we want it to be :-)
I guess I can rotate text, add on :hover events? Paste DIVs on a cube and rotate it?
Good work :-)
monk.e.boy
"SVG has these features but they can only be applied to SVG elements. You can embed the HTML in a , but that has major limitations; in particular, you can't apply effects to HTML elements via stylesheets without changing the markup --- ripping out content and putting it inside ugly SVG fragments --- and breaking most CSS layouts in the process."
Actually its perfectly possible to apply these to other markup languages, provided its clear what the coordinate system is. (Your suggestion on using the union bounding box is reasonable and fully in accord with the SVG definition of bounding box). The new filters module, in particular, takes care to not preclude applying these to other languages and to make clear what a host language must define. And there is of course no requirement to have the SVG in the same document. The properties use URIs not IDREF after all, for a reason.
You also state:
"One approach to solving this is to create new CSS properties for the desired effects.".
Yeah. Another approach is to, you know, use the existing CSS properties, to do the same thing. They are properties for a reason. HTML document A can link to stylesheet B which in turn references SVG filter C.
So while the desire and benefit of applying clip, filter etc to other markup languages is clearly indicated by your post, the necessity to reinvent different, possibly compatible, CSS properties to do so is not at all demonstrated.
Do they work on the output of plug-ins like Flash?
if not, it's just like the old and anti-alias challenged -moz-border-radius, for example.
Great to have but alas not useful for web developers coding for a diverse browser market.
As Henri said, that would get easier with proposed SVG-in-HTML parsing for HTML5. But like I said in the post, the best solution is to support external SVG references.
monk.e.boy: rotating elements is not something I think should be supported through SVG integration. You really want some kind of "CSS transforms" like the Apple proposal.
Chris: I'm deeply confused by your comment. You seem to taking issue with my post, but overall it's clear we agree completely. For example
> the necessity to reinvent different, possibly
> compatible, CSS properties to do so is not at
> all demonstrated.
It should be obvious that I absolutely was not trying to demonstrate that! Quite the reverse, these examples and my implementation use the standard properties and that's my preferred approach. Maybe you should re-read the post.
Sjoerd: I haven't tested non-HTML but yes, this should work on MathML in Gecko since everything but SVG goes through the CSS renderer. On Mac the output of plugins actually is subjected to the SVG effects; on Windows and Linux, only "windowless" plugins are affected.
Andy: yes, you could!
Yes, there is no way to get this working in IE. Yes, it's not currently useful for cross-browser development. The goal is to get this stuff implemented and standardized and exert whatever pressure we can to get IE to follow.
My own opinion on this (as you know since we've talked about it before) is that there should be CSS syntax for simple constructs and that SVG should be used for more advanced capabilities. I think making these properties work on HTML content is a great idea and would like to see both approaches (yours and mine) supported.
I think the trick is where to draw the line when adding features to CSS. It's not clear to me that just because a feature is in SVG that CSS should not do it. However it's also not worth adding a whole lot of super-complex syntax to CSS if we can do something with SVG.
Filters are a great example where the CSS syntax would probably end up being so complex as to not be worth it.
Gradients have a very simple syntax in WebKit's CSS, and I think there's no reason to force the use of SVG for something so simple.
One could also envision overflow-clip and clip being extended in CSS to take a few more basic shapes. They already support rects, so supporting circles, paths, etc. is not much of a stretch and doesn't add much syntactic complexity to CSS. It kind of bugs me that we have similarly named clip properties (clip, clipMask, overflow-clip) in CSS.
To be honest, I prefer the approach of providing general functionality first and then identifying common usage patterns and making them easier, over guessing what the common use cases will be and providing them first.
The case for CSS gradient syntax is particularly strong because authors are using image workarounds already.
Cool stuff, by the way. I generally agree with Hyatt's stance, that I think CSS should have direct support for simple things and applying SVG to HTML allows handling of complex cases where you may need hierarchical constructs to express the style.
Adding SVG path syntax to 'overflow-clip' (or clip-path) could be good if path clipping turns out to be commonly needed. You have to decide what the units of the path should be --- CSS pixels, or fractions of the bounding box? Or do you offer both, using different keywords? You also have to pick a clip-rule (nonzero or even-odd).
The examples above show SVG clip paths, masks, and filter effects. Other things you can gain by integrating HTML with SVG are: fill and stroke HTML text (with gradients and patterns); transformations; animation; timeline synchronization; event transparency (via 'pointer-events'); international text (including vertical); color management; object opacity; group opacity; and graphical cursors. I have probably overlooked as many things as I have listed.
The Mozilla approach is a better direction than attempting to recreate all of the features from SVG as new features of CSS.
Regarding integration of SVG markup as new tags in HTML, that's probably a good thing for the long haul, but likely to result in a long standards and implementation process as that markup integration effort will have tons of complexities. Better to just do something evolutionary and incremental for the time being, such as what Mozilla is showing here.
Regarding getting this stuff working in IE, why don't Firefox or Safari install themselves as IE plugins? The HTML content would need a couple of extra lines to force "render using the plugin", but such an approach would help address the current situation where IE to some extent has veto power over new features.
> with SVG are: fill and stroke HTML text (with
> gradients and patterns);
These probably require new properties, say 'fill-text' and 'stroke-text'.
> transformations;
We should have CSS support for transformations but SVG doesn't really add value here.
> animation; timeline synchronization;
Maybe, but SMIL is horrid.
> event transparency (via 'pointer-events');
Yep.
> international text (including vertical);
Maybe some of the features could be carefully ported over to CSS. Vertical text is already in CSS. Most of the SVG implementation is not reusable though.
> color management;
Yeah.
> object opacity; group opacity; and graphical
> cursors.
In CSS and implemented for a long time.
"The Mozilla approach is a better direction than attempting to recreate all of the features from SVG as new features of CSS."
I don't see anyone trying to recreate all of the features of SVG as CSS features, so I'm not sure who you are directing this to.
Are you kidding us? You haven't seen the need for rounded corners on the web? Isn't that what clipping is, or am I missing the point altogether?
I'm implementing a design that calls for a lot of rounded corners. Creating all the images and hacking in the extra HTML elements and bloating the CSS to achieve such elementary effects is a joke.
Whatever it takes: implementation of border-radius, clipping, whatever!!! PLEASE get the ability to round corners of boxes into the browser market NOW!
Has it already missed the HTML5 IE9/Fx4/SafariX/Opera10 bandwagon? Damn I hope not, or we'll have to wait for the middle of the next decade before we can achieve such an ELEMENTARY effect.
Screw SVG, put this mega-common use-case in CSS!
It's more important than gradients because any designer who uses gradients in that cliche web 2.0 logo reflection technique will stop doing so any minute and will want to create the logos in image format anyway to prevent resizing and maintain the exact presentation of the logo for brand recognition.
Please make my life less depressing and tell me that the east coast American intelligentsia that dictates the web actually realises that such a simple use-case has been around for years and should have been formalised years ago, and like IE8 being a decent browser, it will be at least belatedly fixed very soon.
P.S. last time I checked -moz-border-radius doesn't pass the W3C CSS validator despite the property being named in the custom semi-standard hyphen browser prefix syntax. Why is that?
I think SVG can beat Flash (etc) in the marketplace due to it's ability for strong integration with the document (incl scripting and CSS). I think the success of Flash does show that there is a strong enough desire to create documents leveraging enough graphical features that trying to enhance CSS to provide all that functionality will eventually result in basically reinventing SVG inside of CSS. So, although I don't think I'm in a position to judge all the merits of the competing proposals, I think it would be prudent to have CSS defer to the SVG specifications for defining graphics capabilities whenever possible - don't reinvent the wheel in CSS just for convenience, and instead leverage the existing work in SVG and let it to do what it was designed to do and what it does best.
I have no idea what your point is. border-radius is in CSS3 and WebKit and Gecko both support it. When the spec goes to CR, we'll drop our vendor prefixes.
I was simply pointing out that there is no reason to try to reinvent *all* of SVG in CSS and responding to Jon's implication that we might be trying to do this in WebKit. We're not.
You want to cherry pick the SVG features that will actually become simpler / benefit authors when given CSS syntax. If the feature doesn't get any easier to use when translated to CSS syntax, then it's better to just use SVG.
the following code alerts true in FF2, alerts false in *every* FF3RC I have tried so far.
var SVG = document &&
document.implementation &&
document.implementation.hasFeature &&
document.implementation.hasFeature("org.w3c.dom.svg", "1.0");
alert(SVG);
// FF2 => true
// FF3+ => false
alert(document.implementation.hasFeature("http://www.w3.org/TR/SVG11/feature#CoreAttribute", "1.1"));
So does also the following (of the ones I've tested)
http://www.w3.org/TR/SVG11/feature#Structure
http://www.w3.org/TR/SVG11/feature#BasicStructure
http://www.w3.org/TR/SVG11/feature#ConditionalProcessing
Used the list here
http://www.w3.org/TR/SVG/feature.html
Which of them? The gradient clipping seems like it would require the browsers' cooperation on updating WMODE, and of course browser text is rendered by the browser rather than a common cross-browser extension, but when you categorized the prior few screenfuls of text as "these effects" I wasn't sure which effect in particular prompted the comment.
(Honest question, seeking clarification on your reference.)
"since they're not standards they probably won't ever be invited to this party."
uh-oh, that sounds a little closed and exclusionary there...! :(
tx, jd/adobe
I am being exclusionary, to the extent that I prefer implementation-level and spec-level integration with royalty-free W3C standards over non-standard alternatives.
If Adobe gets Flash accepted as a royalty-free W3C standard (which means giving up total control over its evolution), things would be different. Especially if an open-source implementation was available with friendly licensing.
Got it, thanks for the rephrase. That's true, that the browser renders its own instructions. A plugin could ask the browser to re-render a certain task via NPRuntime:
http://developer.mozilla.org/en/docs/Gecko_Plugin_API_Reference:Scripting_plugins
The opposite holds true as well... JavaScript executed by a browser can ask Adobe Flash Player to re-process and render its own content too, for cross-browser Pixel Bender work:
http://flickr.com/photos/pixelero/tags/pixelbender/
http://www.adobe.com/cfusion/exchange/index.cfm?event=productHome&exc=26
If you wish to wait for W3C, then that's a different situation, true. I'm assuming that Google Gears or realworld video would be off-limits then as well. Your prerogative, agreed.
jd/adobe
Maybe I was confused. However, why isn't dorder radius already in a CR version of CSS? Until M# implements it, we can't use it. If it's not already a real standard, nobody can expect M$ to support it in IE8. Given IE8 will not be released until late this year, will not be installed en mass for longer, we'll have to wait until IE9 at the earliest before we see this simplistic feature.
It drives me bonkers that these things cannot be resolved faster.
The EU has created some light at the end of the tunnel by regulating that M$ become interoperable. In one decision they've achieve more than any Americans can achieve. The time is now to make web standards legally enforceable and therefore the solution to the W3C glacial pace of standards ratification and even slower implementation.
Video's a tough issue because there aren't good unencumbered standards and codecs. In some cases we may have to choose the lesser of evils.
We're on the same page there. Each individual makes their own choices, based on their own criteria.
It was the "won't be invited to this party" line which concerned me, by sounding closed, prejudicial and proprietary. But that sentence had other ambiguities as well, so it's no big deal.
"Now I can party with SVG like people have been partying with Flash" works for me. Have fun with it! :)
jd/adobe
Why we haven't used something like SVG since 1994 makes no sense to me at all, people selling out to Macromedia/Adobe! Even my BBS in the 90s had vector graphics.
Guess I am being very stupid and would welcome some help
many thanks
Could be really useful in things like extensions, where the browser is well-known by the developer!
I'll work with it...
Thank you roc :)
Thanks for the webpage. I still handcode a lot, and I always find .html, .css, .js, .svg, and sometimes, .x3d, .xml, .xsd, and .xsl rearing up to meet me. Add some SQL, a splash of club soda, and presto! Client-side Webpages!
But can someone give me a code example of turning a color jpg into a BW jpg using SVG and/or CSS in Safari?