Sunday, 12 September 2010

"Full Hardware Acceleration"

Microsoft marketing is making noises about IE9 having a monopoly on "full hardware acceleration". They're wrong; Firefox 4 has all the three levels of acceleration they describe. It's surprising they got this wrong, since Paul Rouget published a great overview on hacks.mozilla.org a few days ago (and our plans, source code, and nightly builds have been public since we started working on this stuff many months ago).

One of the messages of their post (and other IE9 marketing materials) is that a browser that only works with D3D and D2D will be superior to one that works on more platforms (e.g. having the option to use GL or D3D for compositing), because they need less of an abstraction layer. They're probably thinking of Chrome, which is basing everything on GL and using ANGLE to map GL to D3D for Windows. We also have an abstraction layer --- which we call "Layers" --- with D3D9, GL and "non-accelerated" implementations, but it's a lot higher-level than GL/ANGLE. (And of course we already have cairo for the content-rendering phase.) It's certainly true that only having to target Vista and Windows 7 would mean less development effort, but an extra abstraction layer need not hurt performance --- if you do it right. I personally like our approach so far; we have to write similar code for GL and D3D often, but having a D3D-specific layers backend makes it easier to integrate D3D-specific features --- like D2D!

I'm pretty confident that our architecture will not cause us any performance disadvantages vs IE9. On the other hand, our architecture does let us deliver D3D acceleration for Windows XP users --- which is still a very large market. I'm surprised Microsoft has been prepared to just hand XP users over to Firefox and Chrome (all those not captive to IE6, at least). Our architecture also lets us deliver GL acceleration for Mac, X, and mobile devices, which is very important.

When we started pushing on Layers at the beginning of the year, I had no idea we'd end up going toe to toe with Microsoft like this. It's exciting!

BTW "full hardware acceleration" is a bogus phrase. All browser pick and choose how to use the GPU, and more use of the GPU isn't necessarily better. There are certainly things no browser is ever going to "hardware accelerate" ... e.g. CSS margin collapsing :-).

In Microsoft's favour, I want to point out that although GPU acceleration for Web graphics isn't going to be a coup for IE9, it is going to be a coup for Windows. As far as I know there simply isn't anything comparable to D2D for GL at this time. (If there was, we'd use it!) cairo-gl is the closest thing I know of that we'd want to use, and but it doesn't seem ready yet. Apple has Quartz 2D Extreme, but they don't enable it for their own apps so one has to assume it's not a clear win. So, kudos to Microsoft for creating D2D, backporting it to Vista, and making it awesome. The free software world really needs to get behind cairo-gl (or something comparable) in a big way. I'd love to have some Mozilla people on it, but we're pretty busy... I'm sure we'd consider funding someone else to work on that stuff, though!



21 comments:

  1. Speaking of which, the current builds are still about 20% slower at Direct2D than the nightly from 20100902. The next day speed plummeted and over time it improved a lot, but it's still not as good (just tested FishIE - 11 vs 13 fps with 1000 fish, 20 vs 24 with 500, 36 vs 42 with 250).
    Since I see bugs 593361 and 593268 marked as fixed, I wonder is this is known. I also wonder what prompted the change that caused this speed loss, as I see no difference in rendering quality, and no speed increase anywhere.
    One last thing. This whole "hardware-accelerated canvas". Canvas is immediate, so I'm assuming on hardware failure (or just driver updates, or maybe hibernation, fast user switching, or even sleep), canvas surfaces could get corrupted and then there wouldn't be a way to restore them (except reloading the page). Is this correct? It also sucks they waste power continuously regardless of visibility.
    Keep up the good work!

    ReplyDelete
  2. Robert O'Callahan12 September 2010 22:06

    The speed drop was turning D3D on, I think. We use D3D9 (so it'll work in XP) and that forces us to do some extra synchronization. I believe Bas has a plan to reduce that overhead.
    I'm not sure what the story is for canvas buffers resiliency. I'm not sure what you mean about "wasting power".

    ReplyDelete
  3. Sorry for being unclear.
    Why is D3D turned on when Direct2D is available? It is my understanding retained layers already worked in Direct2D. Maybe it's for WebGL?
    The "wasting power" meant that the browser has to honor draw requests to canvas even when they're not visible.

    ReplyDelete
  4. Considering the huge market share of non-DirectX mobile phones, I'd say that an abstraction to support different acceleration backends is not just nice, but neccesary.

    ReplyDelete
  5. Tank you Mozilla for your work on Cairo

    ReplyDelete
  6. Jonathan O' Connor13 September 2010 14:58

    Thanks for your informative post. One thing I'm still unclear on is the state of 2D acceleration on Linux. My understanding is that for 4.0, you're working on the infrastructure that's needed for it to happen in the future, but I'm unclear as to how cairo-gl, your Layers work and other technologies fit together. Is there a definitive resource on the state of things, and if not, could you further clarify what needs to be done for us to catch up with the proprietary platforms in this area?

    ReplyDelete
  7. "As far as I know there simply isn't anything comparable to D2D for GL at this time."
    Uhh.. OpenVG?
    No, I know support isn't great yet but people are just as well "getting behind" it than cairo-gl.
    robert.

    ReplyDelete
  8. Lets fix the current blurring font issue, its gonna hurt people's eyes, and it has to be solved. Before that, We can't really claim anything.

    ReplyDelete
  9. Robert O'Callahan14 September 2010 04:09

    Rec: browsers have to honor draw requests to canvases that aren't visible. There's no way around that, since you never know if script might make a canvas visible, and you need to record what the script drew just in case it becomes visible.
    Jonathan: cairo-gl would fit right into our architecture, but it's just not fast enough to use last time I saw numbers. If it just got faster, we could use it.
    Robert: what GPUs+platforms have OpenVG drivers at the moment? Pretty much every non-Microsoft device with a GPU supports GL. I'd heard that OpenVG support was a lot less widespread, but I'd gladly welcome a correction on that.
    chris: not sure exactly what you're talking about.

    ReplyDelete
  10. As fas as I know MS has announced it acceleration only for i series of processors. Any news about FF?

    ReplyDelete
  11. I think chris is talking about the thin, bad looking fonts with D2D/DW, which IE9 also suffers from as it's inherent to D2D/DW. IIRC D2D/DW always uses grayscale anti-aliasing, instead of per color? From what I read on bugzilla, it may be possible to work around in certain situations, but the underlying problem is something Microsoft will have to patch themselves.

    ReplyDelete
  12. So from what I'm reading, the canvas spec etc have no support for callbacks etc to cause drawing only when necessary.
    For 25 years or more, this concept has existed and worked well in gui toolkits etc but now we have to throw it out with nothing better?
    Congratulations. Why oh why does the current generation of retards insist on the devolution of technology? I see it everywhere these days. 98% of laptops sold these days have only 1360x768 screens. iPhones/iPads threw away multitasking.
    firefox 4 is to have even less configuration options.
    all you people do is remove features and add a glossy look and then applaud yourselves. it's just pathetic. If HTML5 was also designed so badly, no wonder no-one will support it.

    ReplyDelete
  13. here is the blur font bug I was talking about:
    https://bugzilla.mozilla.org/show_bug.cgi?id=594325
    I know its a bug of Microfost, but I can't imagine mozilla shipping firefox 4 with this bug while still being responsible for users' eyesight. Its not goona work.

    ReplyDelete
  14. You're correct, Robert: we would definitely consider funding someone to do such work.

    ReplyDelete
  15. As an experienced game developer with 10 years experience writing code for the GPU, I can with complete certainty that Microsoft is going to school you here. Stop whining about poor decisions and either accept that fact that redirection layers are never going to be as fast or start playing the same game as Microsoft. They're going to kick your but. This is coming from a long time Firefox user who happens to know something about what you're doing.

    ReplyDelete
  16. "cairo-gl would fit right into our architecture, but it's just not fast enough to use last time I saw numbers. If it just got faster, we could use it"
    It seems that cairo-drm is already faster -even though probably currently only available on Intel i915-i965 GPUs (so including Atom Netbooks/Tablets etc.!)
    Older benchmarks:
    http://ickle.wordpress.com/2009/08/19/more-performance-wip/
    If could be worth it to implement experimental cairo-drm and/or cairo-gl Backends if it fits nice, naturally after work on 4.0 is done.
    That way people can play with it and it may even accelerate the development of cairo-drm/gl

    ReplyDelete
  17. Robert O'Callahan18 September 2010 17:38

    That might be a good idea.

    ReplyDelete
  18. I find Google's approach to hardware acceleration pretty interesting, mostly because it's quite different from what you & MS are doing. They've done the easy stuff with compositing and canvas, and now are (to my unschooled eye) building their own D2D inside Webkit. (not all landed yet, but what has been is in platform/graphics/gpu if you're curious). It looks like they are taking code from O3D and Skia and so far have got paths working - which maps pretty well to some SVG. That way they can keep using all-OpenGL and get the benefits on all platforms. A lot of work, but they've certainly got the engineers to throw at it and it's a way around the Windows-only problem you talk about. It'll be interesting to see how it works out for them.

    ReplyDelete
  19. Robert O'Callahan19 September 2010 06:49

    Yes, that's what they're doing. It's a fine approach but it's going to get a bit ugly around the edges, for example if they want to match DirectWrite font rasterization, or use any D3D features that don't map well to OpenGL.

    ReplyDelete
  20. I think someone might have started working on a cairo state tracker for gallium at some point too... so it might end up that on Intel GPU's you get cairo-drm and for everyone else you get cairo-gallium

    ReplyDelete
  21. Considering the huge market share of non-DirectX mobile phones, I'd say that an abstraction to support different acceleration backends is not just nice, but neccesary.

    ReplyDelete