Thursday 20 May 2010
WebM
It's a relief to be able to talk about the new WebM video format. Development of WebM support in Firefox has all happened in our Auckland office (with some last-minute build support from Nick down in Timaru, and some quick hacking by Justin Dolske to make the context menu useful on Youtube). It's been crazy and stressful, but it's also been really good because WebM is going to be great for the open Web. There are several major advances here for unencumbered video on the Web:
- Youtube. Youtube supporting WebM is psychologically massive. It also means we won't have to choose between our principles and being the only browser that needs plugins to play Youtube.
- Better video quality. There's a lot of noise from H.264 proponents about Dark Shikari's analysis of VP8, but even he agrees it's better than H.264 Baseline, which is pretty much all H.264 on the Web today (partly because that's all most current H.264 hardware supports). We also will also make ongoing improvements, perhaps by merging in some of the work done for Theora.
- Hardware support. Several companies have announced support for VP8 in their hardware.
- Flash. Support for WebM in Flash may actually be the most important thing for authors and for the success of unencumbered video. It means authors will be able to publish in one format, WebM, and it will work everywhere (except the iPhone/iPad I guess!). Not only is that great for authors, it will put great pressure on the holdouts to add native WebM support. Thanks Adobe!
And thanks Google. Spending $120M on On2 and then releasing their IP assets under a BSD license is hugely appreciated.
It's important to remember that as exciting as this is, it's only the beginning. We expect significant improvements to the VP8 encoder and decoder. For Gecko, we need to get our current patches into mozilla-central, we need to fix some bugs, and we need to add a few desperately needed features to our <video> element support, such as the buffered attribute, plus a generic fullscreen API.
By the way, there's an important lesson here for all the people who were counseling us to give up on unencumbered video as a lost cause and embrace the MPEG-LA. It's a mistake to give in to despair when you don't know the future (and the world being an unpredictable place, you very rarely know the future).
Comments
My inner nerd is disappointed that WebM is immutably frozen to the current unpolished VP8 implementation, but I guess a hobbled, high-quality Free codec is better than an excellent medium-quality Free codec (Theora) or an excellent high-quality patented codec (x264).
Google hasn't provided much in the way of assurances here. They certainly aren't going to indemnify everyone in the world against MPEG LA lawsuits, but I was hoping for at least some discussion of the issue. Instead all we have to go on is Dark Shikari's opinion. I suppose everyone besides him has been muzzled by their lawyers...
For example check out the changes in Theora, or the difference between a good H.264 encoder ( e.g. x264) and a bad one (e.g. Apple or Adobe's). That is all done without spec changes. Since VP8 is currently tuned to meet the misleading PNSR metric there is plenty of low hanging fruit and even right now it is better than most H.264 encoders, it just so happens that the popular GPL encoder x264 is a bit of an outlier when it comes to quality. Probably a good sign for free codecs generally.
On patents: if VP8 turns out to require licensing from the MPEG-LA, then Google wasted most of $120M buying On2. Are they that stupid?
Which is the reason I am sad about https://bugzilla.mozilla.org/show_bug.cgi?id=435339#c53 and eagerly waiting on resolution of https://bugzilla.mozilla.org/show_bug.cgi?id=422540 which could save you a lot of work already.
I don't really like this policy, but that's what it is. Recent threads on mozilla.governance discuss this in more detail.
I wonder though if we have answers to Dark Shikari's analysis, esp. if we have made our lawers think about the patent side and if we are working with Google and others to get the spec to be improved - not sure how much that's possible in terms of what is actually in it but at least it would be good in terms of how it's written (i.e. let's care that it's made crisp and clear and not be expressed in c&p code but clear and useful descriptions).
Well, this is not exactly true. All desktop cards support High profile. The decoder on the iPhone 3GS also supports High. Most "real" devices support high (set top boxes, Blu-Ray players...) YouTube encoded videos use Main for normal qualities (360p, 480p) and High for HD (720p, 1080p) (ironically they don't use B-frames, the lack of which is VP8's greatest problem)
That said, even by DS' admission, VP8 is probably within 20% of High profile h.264, and that's great. The Flash thing is also huge. I'm a lot more optimistic now.
I believe Youtube uses Baseline for 360p.
http://whereswalden.com/2010/05/20/a-brief-note-on-web-video-formats-mozilla-theora-and-h-264-in-light-of-webm/
Also, I never heard them specifically say that they will install the codec on DirectShow (or GStreamer and QuickTime). They haven't done that with any of their other codecs. So IE9 users will still have to install a separate one themselves.
The patent issue has been the elephant in the room that *everyone* is talking about --- it is the *only* reason Google et al. are looking at an alternative. Google would have spent the vast bulk of the time since the On2 purchase going over the source code and algorithms with the finest of fine tooth combs. Nobody, and *least* of all Google, would have taken On2's statements about the patent-free status of VP8 on face value.
Shikari's post makes for a great technical insight into VP8, but he has thrown in his own baseless scaremongering about patent issues. (IANAL indeed!) I can't imagine that he would have had much experience with avoiding patents or patent issues --- especially given the project he works on.
http://www.apple.com/iphone/specs.html
I've heard it said that the newest iPhones could support more in theory, but Apple won't transfer the better files because they want to keep things standard across the installed base. They wouldn't like it if people stopped serving baseline as they've got some 100 million customers who can't view anything better.
The iPad only does Main profile too, so I doubt the next iPhone expected to be launched in the summer will do anything higher than that.
(Wikipedia actually claims that the Mac OS X decoder in Quicktime doesn't even do High Profile, but that's got to be wrong surely?)
But more importantly, even though Youtube is always at the core of this debate (even now it's on the righteous side), it doesn't reflect the average website that just wants to stick a single file on the web and have it viewed everywhere and by everyone. The only contenders for that are H.264 baseline and VP8, and if quality is the deciding factor (which it never is) then VP8 will win.
Another lesson is not "giving in to" hype. The article by Jason Garret-Glaser:
http://x264dev.multimedia.cx/?p=377
gives a much more realistic view than that coming out of the WebM partners and thankfully it's not concluded with a faith-oriented "lesson" suggesting skepticism is a "mistake".
In summary:
- Patent pending? "With regard to patents, VP8 copies way too much from H.264 for anyone sane to be comfortable with it, no matter whose word is behind the claim of being patent-free. This doesn’t mean that it’s sure to be covered by patents, but until Google can give us evidence as to why it isn’t, I would be cautious."
- Inferior quality? "VP8, as a spec, should be a bit better than H.264 Baseline Profile and VC-1. It’s not even close to competitive with H.264 Main or High Profile."
- Closed to developers? "With the lack of a real spec, the VP8 software basically is the spec–and with the spec being “final”, any bugs are now set in stone. Such bugs have already been found and Google has rejected fixes"
Whether Jason is right about quality "as a spec" or not (he is an H.264 partisan after all), in practice we'll improve the VP8 encoder and we think it'll be more than competitive. It's also worth noting that a lot of the devices that play H.264 today only play Baseline.
A real spec for VP8 will be written.
None of that has anything to do with the lesson that unencumbered video has a fighting chance in the marketplace, and those who claimed it was "clearly doomed" would have had us surrender prematurely.
At some point you have to have the courage to take on these patent bogeymen, and fight, and either win or lose.
Do it. You're stronger than the fraidy-cats.
This is GREAT NEWS. If this succeeds, even if or when Mozilla dies, you'll be remembered for this, for decades.
I would like to see more discussion of patents on VP8 from Google's direction since VP8 is newer than things like H.261 and MPEG-1. However time is not on MPEG-LA's side for stopping patent free codecs totally.
I'm guessing that's something else the naysayers thought impossible. It'll be interesting to see what open co-opetition can achieve in VP8 encoders and decoders as the Theora, FFMPEG and x264 projects all seem to have achieved great things within the limitations of their respective specifications.
Crazy, but possible. In a pool with enough patents some of them will be in this category.
A review of 1000+ patents is a huge task, but achievable. My experience of large patent clearance projects suggests that pending applications pose the greatest risk. The continuation and amendment rules in USA (and some other countries) allow too much flexibility to keep something pending and present new claims that target a product that skirts a granted patent. Whether that applies in patent pools, or in this patent pool, I don't know.
Recent cases on enablement may curtail this, or provide some ammunition for litigation. But a patent system that requires full scale litigation to understand the valid scope of a patent is ....
Personally, I like the antitrust angle.
Yes, it is possible in general that there are patents that still cover very old technology because of long times between filing and granting. As you know, for patents filed before July 1995, the length of the patent was the longer of 20 years after filing or 17 years after granting. However, for a known list of patents, it is easy to grab the filing date and granting date and figure out which patents are after a given set of prior art was published.
Here are some scripts I made to automate this process:
http://en.wikipedia.org/wiki/User:Jrincayc/Patent_utils
Personally i dont understand why we cant iron out the bugs and call it something else like VP9. Afterall there are no compatibility issues to worried. Why not fix it before problems spread.
2. Encoder takes a LONG time to tune. It will be at least a year or two before it is into good shape. Ofcoz if google decide to hire all the smart people then it may speed things up.
3. Google has lot of smart guys, as some previous poster have stated. But definitely not about Video Coding. Although they could easily hire lots...
4. What are the chances we could get Current hardware decoder to work with VP8 through software update. I know the chances is slim. But worth asking.
When Google bought ON2, they also bought their entire employee base. On top of that, Google has the ear of XiphOrg and a number of other smaller outfits. Both Mozilla and Opera and a ton of others can contribute code to the project.
I think its way too early to make any declarations. But I suspect that the encoder will improve very quickly.
As for the spec. I suspect that they are still deciding the details.