Friday, 21 May 2010

WebM

It's a relief to be able to talk about the new WebM video format. Development of WebM support in Firefox has all happened in our Auckland office (with some last-minute build support from Nick down in Timaru, and some quick hacking by Justin Dolske to make the context menu useful on Youtube). It's been crazy and stressful, but it's also been really good because WebM is going to be great for the open Web. There are several major advances here for unencumbered video on the Web:


  • Youtube. Youtube supporting WebM is psychologically massive. It also means we won't have to choose between our principles and being the only browser that needs plugins to play Youtube.
  • Better video quality. There's a lot of noise from H.264 proponents about Dark Shikari's analysis of VP8, but even he agrees it's better than H.264 Baseline, which is pretty much all H.264 on the Web today (partly because that's all most current H.264 hardware supports). We also will also make ongoing improvements, perhaps by merging in some of the work done for Theora.
  • Hardware support. Several companies have announced support for VP8 in their hardware.
  • Flash. Support for WebM in Flash may actually be the most important thing for authors and for the success of unencumbered video. It means authors will be able to publish in one format, WebM, and it will work everywhere (except the iPhone/iPad I guess!). Not only is that great for authors, it will put great pressure on the holdouts to add native WebM support. Thanks Adobe!

And thanks Google. Spending $120M on On2 and then releasing their IP assets under a BSD license is hugely appreciated.

It's important to remember that as exciting as this is, it's only the beginning. We expect significant improvements to the VP8 encoder and decoder. For Gecko, we need to get our current patches into mozilla-central, we need to fix some bugs, and we need to add a few desperately needed features to our <video> element support, such as the buffered attribute, plus a generic fullscreen API.

By the way, there's an important lesson here for all the people who were counseling us to give up on unencumbered video as a lost cause and embrace the MPEG-LA. It's a mistake to give in to despair when you don't know the future (and the world being an unpredictable place, you very rarely know the future).



Posted by Robert at 1:08 AM
Labels:

27 comments:

  1. WebM should definitely solve the problem for YouTube, and most other video-sharing sites will likely follow YouTube's lead. Unfortunately, sites such as TV networks and Hulu will probably stick with Flash for the foreseeable future, because the video tag makes it easy for web browsers and other tools to offer downloads of the videos in question, and such sites don't seem to like that idea.
    ReplyDelete
  2. I'm not terribly fussed about video-quality - I've been watching VP6 Flash video online for long enough - so I found the "VP8 is scarily close to the heavily-patented H.264" news from Dark Shikari's analysis to be most worrying.
    My inner nerd is disappointed that WebM is immutably frozen to the current unpolished VP8 implementation, but I guess a hobbled, high-quality Free codec is better than an excellent medium-quality Free codec (Theora) or an excellent high-quality patented codec (x264).
    ReplyDelete
  3. This is all amazing but unfortunately I think it's premature to declare victory; MPEG LA isn't defeated yet. The more worrying part of Dark Shikari's analysis is his opinion that H.264 patents almost certainly cover what VP8 is doing. Even if it isn't true, an MPEG LA lawsuit could drag on for years and kill VP8's momentum.
    Google hasn't provided much in the way of assurances here. They certainly aren't going to indemnify everyone in the world against MPEG LA lawsuits, but I was hoping for at least some discussion of the issue. Instead all we have to go on is Dark Shikari's opinion. I suppose everyone besides him has been muzzled by their lawyers...
    ReplyDelete
  4. @screwtape: VP8 is only "frozen" at the spec level, the nature of most codecs leaves plenty of room for improvement.
    For example check out the changes in Theora, or the difference between a good H.264 encoder ( e.g. x264) and a bad one (e.g. Apple or Adobe's). That is all done without spec changes. Since VP8 is currently tuned to meet the misleading PNSR metric there is plenty of low hanging fruit and even right now it is better than most H.264 encoders, it just so happens that the popular GPL encoder x264 is a bit of an outlier when it comes to quality. Probably a good sign for free codecs generally.
    ReplyDelete
  5. Robert O&#39;CallahanMay 20, 2010 01:10 PM
    I totally agree that victory has not yet been achieved. I fully expect the MPEG-LA to fire back somehow. But I think our chances are looking good.
    On patents: if VP8 turns out to require licensing from the MPEG-LA, then Google wasted most of $120M buying On2. Are they that stupid?
    ReplyDelete
  6. Dave: Implementation bugs have been frozen into the spec. You won't fix those without a whole new revision.
    ReplyDelete
  7. Robert O&#39;CallahanMay 20, 2010 02:07 PM
    The spec constrains the decoder and encoder, but the implementation of the decoder and encoder is by no means "frozen into the spec".
    ReplyDelete
  8. > the world being an unpredictable place, you very rarely know the future
    Which is the reason I am sad about https://bugzilla.mozilla.org/show_bug.cgi?id=435339#c53 and eagerly waiting on resolution of https://bugzilla.mozilla.org/show_bug.cgi?id=422540 which could save you a lot of work already.
    ReplyDelete
  9. Robert O&#39;CallahanMay 20, 2010 02:51 PM
    We don't ship libraries that are LGPL-only. We only ship libraries that are at least LGPL/MPL compatible.
    I don't really like this policy, but that's what it is. Recent threads on mozilla.governance discuss this in more detail.
    ReplyDelete
  10. First, this is great news, and the support behind it is incredibly awesome. This is surely good to see.
    I wonder though if we have answers to Dark Shikari's analysis, esp. if we have made our lawers think about the patent side and if we are working with Google and others to get the spec to be improved - not sure how much that's possible in terms of what is actually in it but at least it would be good in terms of how it's written (i.e. let's care that it's made crisp and clear and not be expressed in c&p code but clear and useful descriptions).
    ReplyDelete
  11. There's actually an interesting question regarding lawsuits here. If the MPEG-LA brings a lawsuit regarding VP8, does that exclude all MPEG-LA members from Google's VP8 license? Or just MPEG-LA itself?
    ReplyDelete
  12. > H.264 Baseline, which is pretty much all H.264 on the Web today (partly because that's all most current H.264 hardware supports)
    Well, this is not exactly true. All desktop cards support High profile. The decoder on the iPhone 3GS also supports High. Most "real" devices support high (set top boxes, Blu-Ray players...) YouTube encoded videos use Main for normal qualities (360p, 480p) and High for HD (720p, 1080p) (ironically they don't use B-frames, the lack of which is VP8's greatest problem)
    That said, even by DS' admission, VP8 is probably within 20% of High profile h.264, and that's great. The Flash thing is also huge. I'm a lot more optimistic now.
    ReplyDelete
  13. Robert O&#39;CallahanMay 21, 2010 02:18 AM
    Anonymous, yes I should have said "most *mobile* hardware".
    I believe Youtube uses Baseline for 360p.
    ReplyDelete
  14. I elaborated my own take on the concern acknowledged in the last paragraph, for anyone interested in reading more words on this topic.
    http://whereswalden.com/2010/05/20/a-brief-note-on-web-video-formats-mozilla-theora-and-h-264-in-light-of-webm/
    ReplyDelete
  15. I think the attention on Dark Shikari's post is that his was the only technical commentary on launch. All of the launch PR was focused on being open-source and royalty-free, and his post was the only one in that void and so is now being taken as fact by the hard-working tech press. I'm quite surprised that 2 days later there hasn't been a technical response from anyone. There's plenty to say; it's the best free codec, it's one of very few codecs that's better than baseline h264, it has plenty of room to grow, etc. But nobody has made any of these points - is On2 just not used to being open yet or something?
    ReplyDelete
  16. I think you're mistaken about Flash supporting WebM. Adobe has only said it supports the VP8 codec not the WebM container format. I think it's likely to be only be supported through a format like FLV or F4V.
    Also, I never heard them specifically say that they will install the codec on DirectShow (or GStreamer and QuickTime). They haven't done that with any of their other codecs. So IE9 users will still have to install a separate one themselves.
    ReplyDelete
  17. What annoys me most about Dark Shikari's post is that, while he's clearly a smart guy, he doesn't seem to understand that Google is staffed by smart guys too.
    The patent issue has been the elephant in the room that *everyone* is talking about --- it is the *only* reason Google et al. are looking at an alternative. Google would have spent the vast bulk of the time since the On2 purchase going over the source code and algorithms with the finest of fine tooth combs. Nobody, and *least* of all Google, would have taken On2's statements about the patent-free status of VP8 on face value.
    Shikari's post makes for a great technical insight into VP8, but he has thrown in his own baseless scaremongering about patent issues. (IANAL indeed!) I can't imagine that he would have had much experience with avoiding patents or patent issues --- especially given the project he works on.
    ReplyDelete
  18. @anonymous: the iPhone 3GS does not support anything higher than H,264 baseline. See for yourself:
    http://www.apple.com/iphone/specs.html
    I've heard it said that the newest iPhones could support more in theory, but Apple won't transfer the better files because they want to keep things standard across the installed base. They wouldn't like it if people stopped serving baseline as they've got some 100 million customers who can't view anything better.
    The iPad only does Main profile too, so I doubt the next iPhone expected to be launched in the summer will do anything higher than that.
    (Wikipedia actually claims that the Mac OS X decoder in Quicktime doesn't even do High Profile, but that's got to be wrong surely?)
    But more importantly, even though Youtube is always at the core of this debate (even now it's on the righteous side), it doesn't reflect the average website that just wants to stick a single file on the web and have it viewed everywhere and by everyone. The only contenders for that are H.264 baseline and VP8, and if quality is the deciding factor (which it never is) then VP8 will win.
    ReplyDelete
  19. The "lesson to be learned" is that Google cash bankrolls the vast majority of 'open' ideology. It is not "giving in to despair" to acknowledge that "open" is not always pragmatic.
    Another lesson is not "giving in to" hype. The article by Jason Garret-Glaser:
    http://x264dev.multimedia.cx/?p=377
    gives a much more realistic view than that coming out of the WebM partners and thankfully it's not concluded with a faith-oriented "lesson" suggesting skepticism is a "mistake".
    In summary:
    - Patent pending? "With regard to patents, VP8 copies way too much from H.264 for anyone sane to be comfortable with it, no matter whose word is behind the claim of being patent-free. This doesn’t mean that it’s sure to be covered by patents, but until Google can give us evidence as to why it isn’t, I would be cautious."
    - Inferior quality? "VP8, as a spec, should be a bit better than H.264 Baseline Profile and VC-1. It’s not even close to competitive with H.264 Main or High Profile."
    - Closed to developers? "With the lack of a real spec, the VP8 software basically is the spec–and with the spec being “final”, any bugs are now set in stone. Such bugs have already been found and Google has rejected fixes"
    ReplyDelete
  20. Robert O&#39;CallahanMay 24, 2010 04:28 AM
    Jason is deeply invested in H.264, but he doesn't know much about patents (his project totally ignores patents). The most important point about patents is that if VP8 turns out to require MPEG-LA patent licensing, Google wasted most of $120M buying On2. One presumes they did due diligence.
    Whether Jason is right about quality "as a spec" or not (he is an H.264 partisan after all), in practice we'll improve the VP8 encoder and we think it'll be more than competitive. It's also worth noting that a lot of the devices that play H.264 today only play Baseline.
    A real spec for VP8 will be written.
    None of that has anything to do with the lesson that unencumbered video has a fighting chance in the marketplace, and those who claimed it was "clearly doomed" would have had us surrender prematurely.
    ReplyDelete
  21. Every attempt to bring an open patent/royalty free video platform for the web will come up for criticism that it will be sunk by a patent aircraft carrier or submarine.
    At some point you have to have the courage to take on these patent bogeymen, and fight, and either win or lose.
    Do it. You're stronger than the fraidy-cats.
    This is GREAT NEWS. If this succeeds, even if or when Mozilla dies, you'll be remembered for this, for decades.
    ReplyDelete
  22. For a royalty free video platform, H.261 like video (note the 1, not a 3 or a 4) and somekind of subband audio encoding similar to MASCAM is rapidly approaching the point where it is indisputably not covered by any MPEG-LA patents simply by age since H.261 is from 1990 and MASCAM is from 1988. There is at least one MPEG-LA MPEG-2 patent expiring every month for the rest of this year ( http://en.wikipedia.org/wiki/Talk:MPEG-2#Patents ) In about two years, MPEG-1 should be approaching this same point since the near final public committee draft came out by December 6, 1991. So claims like all video codecs are subject to MPEG-LA's patents are rapidly becoming false even in the unlikely event that they are currently true.
    I would like to see more discussion of patents on VP8 from Google's direction since VP8 is newer than things like H.261 and MPEG-1. However time is not on MPEG-LA's side for stopping patent free codecs totally.
    ReplyDelete
  23. I can't help but reflect on the progress that Javascript execution speed has made over the last few years as Google, Mozilla, Opera and Apple (even Microsoft more recently) have worked diligently, sometimes in co-operation, sometimes in competition to squeeze any possible speed increase out of the already existing standard.
    I'm guessing that's something else the naysayers thought impossible. It'll be interesting to see what open co-opetition can achieve in VP8 encoders and decoders as the Theora, FFMPEG and x264 projects all seem to have achieved great things within the limitations of their respective specifications.
    ReplyDelete
  24. US patent rules prior to July 1995 mean that some patents could extend out to 2017 or so, even though they cover standards from 1990.
    Crazy, but possible. In a pool with enough patents some of them will be in this category.
    A review of 1000+ patents is a huge task, but achievable. My experience of large patent clearance projects suggests that pending applications pose the greatest risk. The continuation and amendment rules in USA (and some other countries) allow too much flexibility to keep something pending and present new claims that target a product that skirts a granted patent. Whether that applies in patent pools, or in this patent pool, I don't know.
    Recent cases on enablement may curtail this, or provide some ammunition for litigation. But a patent system that requires full scale litigation to understand the valid scope of a patent is ....
    Personally, I like the antitrust angle.
    ReplyDelete
  25. Joshua CogliatiMay 26, 2010 06:43 PM
    Logan:
    Yes, it is possible in general that there are patents that still cover very old technology because of long times between filing and granting. As you know, for patents filed before July 1995, the length of the patent was the longer of 20 years after filing or 17 years after granting. However, for a known list of patents, it is easy to grab the filing date and granting date and figure out which patents are after a given set of prior art was published.
    Here are some scripts I made to automate this process:
    http://en.wikipedia.org/wiki/User:Jrincayc/Patent_utils
    ReplyDelete
  26. 1. Unless google comes out and say. VP8 Spec will be written with Bug Fixing. Otherwise the reference Encoder ( with bug ) as it is now, will be the only encoder. This isn't necessarily a bad thing. But as a encoder that On2 has been working for a few years it is in VERY bad shape.
    Personally i dont understand why we cant iron out the bugs and call it something else like VP9. Afterall there are no compatibility issues to worried. Why not fix it before problems spread.
    2. Encoder takes a LONG time to tune. It will be at least a year or two before it is into good shape. Ofcoz if google decide to hire all the smart people then it may speed things up.
    3. Google has lot of smart guys, as some previous poster have stated. But definitely not about Video Coding. Although they could easily hire lots...
    4. What are the chances we could get Current hardware decoder to work with VP8 through software update. I know the chances is slim. But worth asking.
    ReplyDelete
  27. Google was never about anything but search. But now they are practically everything.
    When Google bought ON2, they also bought their entire employee base. On top of that, Google has the ear of XiphOrg and a number of other smaller outfits. Both Mozilla and Opera and a ton of others can contribute code to the project.
    I think its way too early to make any declarations. But I suspect that the encoder will improve very quickly.
    As for the spec. I suspect that they are still deciding the details.
    ReplyDelete

Subscribe to: Post Comments (Atom)