Friday, 11 March 2011

The Myth Of "Full Hardware Acceleration"

A browser that "fully hardware-accelerates the entire Web platform" shouldn't have any trouble with a simple HTML scrolling testcase, should it? Try this testcase in the browsers on your machine. Running fullscreen on my laptop, IE9 RC1 takes more than ten times as long as Firefox 4. That doesn't seem very accelerated to me. View the source to see that the testcase does nothing tricky.

When we designed our "layers" framework for GPU acceleration in Firefox, one of the goals was to accelerate scrolling, especially on pages where some parts of the page move as you scroll and other parts don't --- like that testcase. We have a component called FrameLayerBuilder that separates the moving parts from the stationary parts and assigns them to separate "layers". (The vicissitudes of CSS make this surprisingly difficult, but that's a topic for another blog post...) Each layer gets a VRAM buffer into which we draw the layer's contents; when we scroll, we only have to draw the content that has scrolled into view, and then we use the GPU to composite the layers together at their new positions.

I can't say for sure what IE9 is doing in this case, but experimentation suggests that it simply repaints the entire page from scratch every time it scrolls the above testcase. This does not make best use of the hardware.

I'm not here to bash IE9, just its marketing. Microsoft's message that IE9 is the apex of what a browser can do with the GPU is nonsense; in the above example, Firefox's rendering architecture makes better use of the GPU, by design. Microsoft's claim to hardware-accelerate "the entire Web platform" was always ridiculous because "the entire Web platform" contains features like HTML parsing, Javascript execution, and DOM manipulation which are simply not suitable to run on today's GPUs, and IE9 does not do so. (I'm assuming that Microsoft's "hardware acceleration" means "uses the GPU" ... if it actually means "uses hardware", that isn't saying much :-).) Microsoft's disdainful statement that "[other browsers are] choosing to accelerate only pieces of the Web platform" is hypocritical, because that's exactly what they do themselves.



38 comments:

  1. Lenovo T410 with GMA HD and up to date drivers/WinXP.
    Firefox 4 B12 with h/w accel enabled: 14348ms
    Chrome 11.0.696.0 DEV: 8698ms

    ReplyDelete
  2. Robert O'Callahan11 March 2011 12:43

    Since you asked...
    On my laptop, on battery since I don't have my charger at home:
    Firefox 4 RC1 with D3D9: 7s
    Firefox 4 RC1 with no GPU acceleration: 8s (the layers stuff I mentioned works pretty well even if we just use the CPU)
    Firefox 4 RC1 with D3D10: 9s
    Chrome 11.0.696.0: 25s
    IE9: 190s
    No idea why the results for Firefox and Chrome are so different to yours. However, that was not the point of this post.

    ReplyDelete
  3. I haven't had a chance to fiddle/play around with FF4 too much yet and I hate to keep making suggestions,but why not have the URL place (real techie talk;)be a mouseover instead of taking up room??Or maybe someway to rearrange location like on the vertical? just thinking out loud. I'm constantly awed by the creativess;)

    ReplyDelete
  4. Have you checked that IE9 isn't using Software Rendering Mode, as per http://blogs.msdn.com/b/ieinternals/archive/2011/02/17/ie9-benchmark-tests-and-gpu-fallback-to-software-rendering-mode.aspx

    ReplyDelete
  5. All this would be great if this Layers framework really worked. Unfortunately it doesn't and instead breaks functionality from Firefox3 https://bugzilla.mozilla.org/show_bug.cgi?id=637597

    ReplyDelete
  6. Pascal Chevrel11 March 2011 13:57

    On my Linux laptop, Firefox 3.6 is much faster at this test than the latest nightly:
    3.6: 1995mn
    4.0b13pre: 3838ms
    Should I file a bug?

    ReplyDelete
  7. Robert, can you add -webkit and -moz versions of box-shadow and border-radius? That way, people can compare Firefox 3.6, Safari 5, etc.

    ReplyDelete
  8. Pascal Chevrel11 March 2011 14:11

    ok, the perf difference is caused by border-radius and box shadow that do not have prefixes for 3.6.
    With prefixed border-radius and box-shadow:
    3.6: 16108ms
    4.0: 3795ms
    Without border-radius and box-shadow:
    3.6: 2435ms
    4.0: 1494ms
    You should update the css with the various vendor prefixes so as that all browsers are on an equal testing ground :)

    ReplyDelete
  9. Card/Driver: GTX295/260.19.06-0ubuntu1
    Browser: Chrome 9.0.597.107 (Official Build 75357)
    Resolution: 1920x1080
    Time: 2153ms

    ReplyDelete
  10. OK, so Firefox 4 excels at some things and IE9 at others - neither has a perfect rendering engine. IE9 gets the same or better score on your own HW stress test. http://www.conceivablytech.com/wp-content/uploads/2011/03/stress.jpg Scrolling is just one area where IE9 is lacking - thanks for pointing that out. Can't the browsers just live in peace and harmony?

    ReplyDelete
  11. What I find odd is FF 3.6.15 is in the 1.8s range, while FF4rc1 is in the 5.8s range. I find it strange that FF3 is 3x times faster (and I have a bunch of open tabs in FF3 and nothing in FF4rc.)

    ReplyDelete
  12. Don't want to spoil the party but Safari 5.0.3 consistently scores around 4300ms.

    ReplyDelete
  13. My system shows quite different results.
    CPU: i7 920 (2.67 GHz)
    Graphics: Radeon HD 3600
    Firefox 4 RC1: 12s
    [Do not know how to switch between D3D9/10 and off.]
    Firefox 3.6.14: 1414s
    IE 9: 5s [running in a Win7 VM]
    IE 8: Error

    ReplyDelete
  14. Jorde Vorstenbosch11 March 2011 16:02

    Chrome:
    Test #1: 3123ms
    Test #2: 3173ms
    Test #3: 3242ms
    Firefox:
    Test #1: 10788ms
    Test #2: 5532ms
    Test #3: 5579ms
    Safari:
    Test #1: 4696ms
    Test #2: 4337ms
    Test #3: 5031ms
    Latest Versions
    This is on a macbook 5.1 running OS X 10.6.6

    ReplyDelete
  15. Chrome 9.?.?.?: 18898ms
    Chrome 10.0.648.127: 16715ms
    Chrome 11.0.696.0: 17360
    Minefield 4.0b12 (2011-02-22) with D3D10: 6387ms
    Minefield 4.0b13 (2011-03-03) with D3D10: 6361ms
    I have not installed IE9 because of having to rebuild my PC due to a previous installation so no stats there :o/
    This really shows how well the layers stuff works, amazing!

    ReplyDelete
  16. I guess screen resolution also makes a diff?
    2048 x 1152:
    IE9 latest: 104,855ms
    Chrome Canary: 18,542ms
    Chrome 10: 17,989ms
    Firefox 4 RC: 11,825ms
    1280 x 1024
    Firefox 4 RC: 6,051ms
    (Didn't test the rest in that resolution)

    ReplyDelete
  17. Not too realistic a test for average website performance; I don't think it's common for websites to have 10 thousands elements on a single page.
    But, it seems a reasonable test of a browser's rendering architecture efficiency.
    My system:
    Intel Core 2 Duo w/integrated gfx (NVIDIA 9300)
    Ubuntu 10.10 64bit w/NVIDIA 64-bit linux driver
    Yes, FF4 hw acceleration does work with this config; WebGL works as well.
    And because it matters, the size of the viewports I tested were roughly 1152x720.
    (rounded averages from several runs)
    FF 4.0rc : 2s
    FF 3.6.15 : 1.3s
    Chrome 11.0.686.3 : 3.4s
    Opera 11.01 : 25.8s
    Look at FF3.6! Unfortunately, the FF3.6 test is skewed because the border-radius and box-shadow CSS properties didn't have the -moz prefix and so were ignored; web pages can render a lot faster when not applying rounded borders and shadows to over a hundred on-screen elements.

    ReplyDelete
  18. I don't know what these test cases do, but (consistently):
    1. Firefox 3.6.15: 2200ms (approx)
    2. Chrome 8.0.552.237 (that's Chrome's version number, not my IP address): 4500ms (approx)
    Lenovo X200, Windows XP, running on AC power.

    ReplyDelete
  19. Robert O'Callahan11 March 2011 21:51

    As others have noted, the FF3.6 results are meaningless because the unprefixed border-radius and box-shadow are not supported, and they're key to this test. The same may be true for Safari 5 but I can't check right now --- look closely at the page for shadows and rounded corners.
    I didn't add prefixed border-radius or box-shadow because you just know someone is going to see "-moz" in the test and conclude that it's rigged somehow.
    A large window size does show the performance differences more clearly. My windows were fullscreen on a 1920x1080 screen, minus the Windows 7 taskbar on the left and each browsers' chrome at the top.
    Of course IE9 is using acceleration on my system :-).

    ReplyDelete
  20. From visual inspection only, Safari 5.0.4 seems to support border-radius but not box-shadow, so its results are almost as meaningless as Firefox 3.6.

    ReplyDelete
  21. Richard Ayotte12 March 2011 00:22

    Specs: AMD Phenom 1090T, nVidia GeForce GTX 460 (using the nVidia drive ver 270.29), 8G RAM.
    Firefox 4.0b13pre (2011-03-10) on Ubuntu 10.10.
    ---> 87,974ms, 93,106ms and the whole browser is very slow.
    Chrome 11.0.686.3 (76659) Ubuntu 10.10
    ---> 1978ms, 1883ms, 1911ms.
    What's going on here? Should I be heading to bugzilla?

    ReplyDelete
  22. Any comment as to why FireFox4 can't keep up with IE9 on these sort of tests:
    http://blogs.msdn.com/b/ie/archive/2011/03/08/comparing-hardware-accelerated-svg-across-browsers-with-santa-s-workshop.aspx

    ReplyDelete
  23. Robert O'Callahan12 March 2011 02:46

    Yes. I commented in that blog, and I'm writing a new post for my own blog explaining some of these things.

    ReplyDelete
  24. FF 4b13pre (64 bit, Win7): 3743ms
    IE9 RC (64 bit, Win7): 59537ms
    Google Chrome (32 bit on 64 bit Windows 7): 5624ms

    ReplyDelete
  25. Well, IE9 RC still seems to ourperform FF4 RC in the benchmarks: http://bit.ly/g8SbSc

    ReplyDelete
  26. Lately mozilla just continues to criticize IE9, shouldn't be better that you keep focus on making mozilla better?
    I mean guys really, mozilla performs quite bad compared to IE9 in many demos that look much more real than a bunch of circles in a page. I would love to read articles talking on how you work to make it better instead of just saying it already does good or that IE9 does not or it is not modern...
    Keep up the good work and stop talking about other browser (or whatever marketing campaign)

    ReplyDelete
  27. How exactly does taking a few minutes to make a counter argument mean that Mozilla isn't focused on making Firefox better? That's making a pretty big logical leap.
    In fact, roc's next post makes it pretty clear that Mozilla is working quite hard on analyzing their performance in the benchmarks specifically to do better. However, as they've found, some of the benchmarks aren't what they claim to be and I think Mozilla has every right to say that!

    ReplyDelete
  28. >I mean guys really, mozilla performs quite bad compared to IE9 in many demos that look much more real than a bunch of circles in a page.
    So if that roc's demo had some fancy design you would believe that IE9 is worse?
    And debunking myths of competitiors is an important part of marketing. Do you know that IE9 cheats on SunSpider benchmark?

    ReplyDelete
  29. mandriva 2010.2 x86_64
    kernel 2.6.36.2-desktop-2mnb
    Intel(R) Core(TM)2 Duo CPU E8200 @ 2.66GHz
    4 gb ram, nvidia g220
    Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.15) Gecko/20110307 Mandriva Linux/1.9.2.15-0.1mdv2010.2 (2010.2) Firefox/3.6.15
    --> result : 2690ms
    Chromium 11.0.657.0 (73275) Mandriva 2010.2
    --> result : 4421ms
    HTH

    ReplyDelete
  30. Note that your own marketing also uses the term "full hardware acceleration"...
    http://www.mozilla.com/en-US/firefox/RC/features/

    ReplyDelete
  31. @RyanVM
    > So if that roc's demo had some fancy design you would believe that IE9 is worse?
    I'm not talking about a fancy design, I'm talking about games like http://ie.microsoft.com/testdrive/Performance/Helicopter/Default.xhtml or http://www.pirateslovedaisies.com/ and any other demo that goes further than just scrolling thousands of circles
    > Do you know that IE9 cheats on SunSpider benchmark?
    Oh please, don't go that way. What IE9 does is a very common practice on any compiler. Anybody with little knowledge about compilers/jitters design know that this is a standard optimization.
    @Adrian:
    > Note that your own marketing also uses the term "full hardware acceleration"...http://www.mozilla.com/en-US/firefox/RC/features/
    This is funny, I guess it will be removed soon after this post

    ReplyDelete
  32. Thinkpad T23, mixture of Debian Unstable and Ubuntu Natty, 1.2 GHz PIII-m, 768 MB, no GPU worth speaking of (SuperSavage).
    Chromium 11.0.696.0: 9155 ms
    Minefield 4.0b13pre: 7133 ms

    ReplyDelete
  33. Will firefox ever support more acceleration than just WebGL on Windows XP?

    ReplyDelete
  34. This issue was documented and confirmed by Microsoft:
    The 64bit version has not been speed optimized yet. Most of the resources have been committed to the more popular 32bit version.
    source:
    http://www.zdnet.com/blog/networking/chrome-10-vs-internet-explorer-9-reconsidered/792

    ReplyDelete
  35. Robert O'Callahan16 March 2011 20:41

    pd: Yes.
    Jaspio: I was testing the 32bit version.

    ReplyDelete
  36. "Try this testcase in the browsers on your machine"
    A testcase in a transitional mode doctype.
    Are you serious?
    To compare with IE9 at least force a standards doctype either trough either HTML or strict.

    ReplyDelete
  37. Robert O'Callahan19 March 2011 21:51

    That testcase uses a standards-mode doctype.

    ReplyDelete
  38. op:sorry to have to correct you but did you try hard to test ie9!i ll give you a hint!activate compatibility in ie9 and retest,since ie9 hasnt changed since it was lunched it is no cheat,according to my informal test minefield did your test in 5000(+- 700)ie9 did it in 1500((+-700)so as you can see ie 9 wasnt an issue,true if you disable compatibility thing look different but it is hardly slow it is just that some stuff used in that test is either not recognised by ie9 or is very old software.wouldnt have posted but my friend told me it needed to be posted so you would need these info for accurate comparision

    ReplyDelete

Note: only a member of this blog may post a comment.