Eyes Above The Waves

Robert O'Callahan. Christian. Repatriate Kiwi. Hacker.

Friday 26 November 2010

Measuring FPS

It seems standard practice for Web authors to measure performance in FPS (frames per second). There are two large problems with this.

First, FPS is meaningless above 60 or so, because you normally can't get more frames than that onto the screen. A well-designed software stack will actually prevent FPS from rising above 60 --- those extra frames are wasting power. More than 60 FPS means you have a bug (or you are doing very specialized full-screen gaming --- i.e. not a Web browser). I discussed this in more detail here.

Second, Web authors normally measure FPS as "how many times did my event handler run per second". But what they actually should be measuring is how many different frames the browser was able to paint to the screen per second, and that's a very different thing, since a setTimeout handler (for example) can run multiple times between browser repaints. It's understandable that Web authors would make such a mistake, since browsers haven't provided a way to measure the real FPS. However, in Firefox 4 we have added an API for this: mozPaintCount. Use it!

I think authors need to get into the habit of measuring FPS using mozPaintCount or something similar on other browsers and determining first whether their app is able to reach 50-60FPS. If it can, then further performance improvements can only be measured by increasing the complexity of the scene until FPS starts dropping off. Microsoft's FishIE Tank is the right approach. (I don't know how they try to measure FPS though; they're probably not using mozPaintCount, so it's probably not very accurate.)


Anthony Ricaud
Maybe mozPaintCount could be proposed to the Web Performance Working Group http://www.w3.org/2010/webperf/
but what is equivalent to mozPaint for other browsers? i couldn't find anything similar..
Robert O'Callahan
Anthony: that's a good idea.
I don't know if they have anything equivalent.
Once you get anywhere close to 60 fps, the most interesting aspect is not fps but number of screen refreshes for which there was no update.
If you have any kind of fluid motion (say, scroll) and you skip a frame, it looks jerky. This is not clearly visible in fps counts.
Robert O'Callahan
That's a good point. We have no direct way of measuring that right now.
Once again, I'd like to point you out to my jquery interval bookmarklet http://lrbabe.github.com/jquery-interval-bookmarklet/
The demo scene is comparable to the IE fish tank, as it stresses the browser enough to go below 60fps. The main difference is that you are seeing pure DOM animations instead of canvas based ones.
You'll be able to see the number of JS runs per second (fps) as well as the number of paints per second (pps, Mozilla only).
Moreover you'll be able to switch the animation logic to use a "mozRequestAnimationFrame" based one, and see how it affects performances (turn on your system monitor and watch what happens when you switch to another tab).
This bookmarklet can be used on any website using jQuery 1.4.3+
I would also like to point out that the number of js runs per second is still an important stat, especially if it is lower than the number of paints per second, "since a repaint can happen multiple times between setInterval handler". Isn't it?
Ok, so once I see that my mozPaintCount is lousy, can I have mozNeedlessReflowsCount to fix it? ;)
It might make sense to propose something similar to mozPaintCount to WebKit folks (under webkit-prefixed name). I would think it shouldn't be hard to hook internal repaint event to a DOM one. By the way, why is it called "paintCount" and not "onpaint" or "onrepaint" (for consistency; similar to, say, recently introduced "onbeforescriptexecute"/"onafterscriptexecute")?
Robert O'Callahan
kangax: because it's not an event.
Actually I was wrong in my response to sysKin. If you check mozPaintCount every time your animation script runs, you can tell if there was a repaint since the last time your script ran. Ideally paintCount would increase by exactly 1 every time your animation script runs. You won't actually get that on Firefox trunk today, for various reasons, but that's the goal.
I've created a bug in Chrome bug tracker to ask for window.webkitPaintCount