Sunday, 24 January 2010

Video, Freedom And Mozilla

Note: below is nothing but my own opinion as a developer of video-related Mozilla code!

My LCA talk on Friday was about why open video is critically important to free software, and what Mozilla is doing about (plus a discussion of the relationship between Web standards and free software in general). Little did I know that Youtube and Vimeo would pick the day before my talk to cast a glaring spotlight on the issue!

Youtube and Vimeo have started offering video playback using the HTML5 <video> element. That is good news for free software, since it means you don't need a closed-source Flash player to play the video [1]. However, they only offer video in H.264 format, and that is not good news for free software. A lot of people have noticed that Firefox doesn't support H.264, and apparently many people don't understand why, or know what the problems are with H.264. This is a good time to restate the facts and re-explain why Firefox does not support H.264. I'll be mostly recapitulating the relevant chunks of my talk. (Hopefully a full recording of my talk will become available from the LCA site next week.)

The basic problem is simple: H.264 is encumbered by patents whose licensing is actively pursued by the MPEG-LA. If you distribute H.264 codecs in a jurisdiction where software patents are enforceable, and you haven't paid the MPEG-LA for a patent license, you are at risk of being sued.

So why doesn't Mozilla just license H.264 (like everybody else)? One big reason is that that would violate principles of free software that we strongly believe in. In particular, we believe that downstream recipients of our code should be able to modify and redistribute it without losing any functionality. This is freedom that copyleft licenses such as the GPL and LGPL (which we use for our code) are intended to ensure. It is possible to obtain patent licenses in a way which works around the letter of the GPLv2 and LGPLv2, but honoring the letter while violating the spirit is not a game we are interested in playing.

But aren't there (L)GPL implementations of H.264? Yes, but they're not as free as they appear. Their freedom has been silently stolen by patents (in jurisdictions where those patents exist and are enforceable). The software license permits you to redistribute and use the code, but the MPEG-LA can still stop you. [2]

But the MPEG-LA won't bother suing me or my project, we're not worth bothering with. Perhaps true, but I hope "remain irrelevant" is not the favoured strategy for most free software projects. It's certainly not an option for Mozilla. If we hadn't distributed Firefox to tens of millions of users --- legally --- it probably wouldn't be possible to browse the Web today using anything but IE on Windows. Plus, relying on selective enforcement is rarely a good idea.

Mozilla should just ship without licensing as a civil disobedience measure. That might be fun, but I expect an injunction would quickly force us to disable H.264 and send a hefty damages payout to the MPEG-LA. That's not a win.

Mozilla should pick up and use H.264 codecs that are already installed on the user's system. I've previously written about a variety of reasons this would be a bad idea, especially on Windows. Really there are two main issues:


  • Most users with Windows Vista and earlier do not have an H.264 codec installed. So for the majority of our users, this doesn't solve any problem.
  • It pushes the software freedom issues from the browser (where we have leverage to possibly change the codec situation) to the platform (where there is no such leverage). You still can't have a completely free software Web client stack.

But I could just download gst-plugins-ugly and I'd be OK. That's a selfish attitude. Everyone should be able to browse the Web with a free software stack without having to jump through arcane hoops to download and install software (whose use is legally questionable).

The H.264 patents will expire soon, and then we'll be OK. Many H.264 patents don't expire until 2017 or later. Anyway, H.264 isn't the last word in video compression. There will be an H.265 and the same set of problems will persist.

Users just want video to work. You Mozilla people are such idealists! Yes, that is the reason for Mozilla to exist. Anyway, in the short term, our users probably won't be affected much since Flash fallback will still work. In the long term, I think freedom will ultimately benefit users (not just Firefox users, but all users).

Apart from the issues with H.264 support in clients, there are also huge issues around H.264 for Web authors and content providers. Currently providing H.264 content on the Internet is zero-cost, but after 2010 that will almost certainly change. A couple of good articles are here and here. We won't know much about the terms until the end of this month. The key issue is not exactly how much it will cost, but that if you want to publish H.264 you will probably have to hire lawyers and negotiate a license with the MPEG-LA. If you just want to put a few videos on your Web site, or add a help video to your Web application, or put a video cut-scene in your Web game, that is probably not something you want to do. Web video is not just about Youtube; mandatory licensing would cripple the use of video on the Web. (Just imagine if we had such a regime for still images...) Even if there were no patent issues on the client side, this would still be a good reason for Mozilla to push for truly free codecs.

The honest truth is that none of us know how this is going play out. The proponents of mandatory licensing are strong, and most people don't care about software freedom. We're doing our best to make Ogg Theora rock, and I don't know what else we can do directly right now except spread the word and help people understand what's at stake here ... hence my LCA talk, and this blog post.

[1] Yes, I know gnash and/or swfdec might play these videos, but in general they are not able to keep up with the latest Flash APIs offered by Adobe and used by major sites. Anyway, it's good to not have to play that game.

[2] RANT: in many cases, free implementations of heavily patent-encumbered technology are harmful to the free software ecosystem, for two reasons: people are confused into thinking they have rights that they actually don't, and thus these implementations can discourage the adoption of alternatives that are free for everyone.



131 comments:

  1. And unfortunately, the response of a lot of people will still be...
    "Yeah, but H.264 is better than Theora, so they should implement it"

    ReplyDelete
  2. Do you think Ogg is not patent encumbered ? That's forgetting how bad/stupid patents are granted. Don't forget even the wheel is patented. Several times, even.
    But, H264 has known patents, and even worse, they are known to be enforced.
    That also doesn't mean everything else is patent-safe. That's hiding your head in the sand.

    ReplyDelete
  3. Why not stick it to US and use H.264 in localized versions of Firefox?
    I live in a country free from software patents and I don't want to suffer from another country's legislative problems.

    ReplyDelete
  4. I don't understand why there's so much emphasis on Ogg Theora and seemingly no attention paid to Dirac/Schrodinger...

    ReplyDelete
  5. "Users just want video to work. You Mozilla people are such idealists! Yes, that is the reason for Mozilla to exist. Anyway, in the short term, our users probably won't be affected much since Flash fallback will still work. In the long term, I think freedom will ultimately benefit users (not just Firefox users, but all users)."
    Flash fallback. So it's ok to force people to go download a non-free plugin from Adobe, but not ok to force them to go download a non-free H.264 codec. Yeah, I know about things like gnash, nobody uses them, and they suck. Is Firefox going to ship with gnash included already? It will ship with H.264 before it does that.

    ReplyDelete
  6. Other issues aside, I don’t understand why supporting a proprietary Flash plugin from a single vendor is better than opening support for a standardized (albeit similarly patent-encumbered) video format with open-source implementations.

    ReplyDelete
  7. Keep fighting the good fight.

    ReplyDelete
  8. since Flash fallback will still work
    Ah, you see, a problem there is you think flash fallback works... Viewing a youtube video reliably crashes 64-bit firefox (or rather the 64-bit flash plugin, so mozilla just doesn't want to know, but it's in the same process, so also takes out firefox). For the love of god, please just run plugins in separate subprocesses, a bunch of silicon in a modern CPU exists to provide an MMU for interprocess memory protection! Use it!

    ReplyDelete
  9. I love Mozilla for its decision to promote free codecs ans then free softwares. Keep on making good decisions for all

    ReplyDelete
  10. Howdy. My name is Brian Boyko - I'm heading to NZ in... jeez, 24 hours - to start up a job with a company called Template Digital, mostly dealing with motion graphics in the proprietary program of AfterEffects.
    First, as soon as I get settled, mind if I look you up to talk to you about this issue? You can reach me at my address above.
    For the record, I like OGG - and moving away from flash-based to browser-based video will go a long way towards pushing the standard. The other half of the equation, though, is in encoders. My Non Linear Editor (NLE) is Sony Vegas; problem is, I can't figure out how to render in OGG. Pretty much every NLE is proprietary except Cinelerra and that doesn't have features I need. Do you think it's better to develop OGG plugins to proprietary software or wait for OSS NLEs to mature?

    ReplyDelete
  11. Indeed, what if GIF were a patent nightmare!
    Oh, wait, it was.

    ReplyDelete
  12. Thank you for speaking out against H.264 an in support of Ogg/Theora. This has already been poste to reddit but I will be sure to tweet a link to your article.
    Also I made sure to tell google (through feedback) that I am in support of Ogg/Theora instead.
    I sugguest others do the same.

    ReplyDelete
  13. MrNightLifeLover24 January 2010 18:32

    > Mozilla should pick up and use H.264 codecs that are already
    > installed on the user's system.
    Yes you should. The reasons u gave not to use it sound like they come from free software extremist like Stalin.. errr Stallman.
    Of course a problem would be that it's not available on Win XP, however it would work on Win 7 and Vista. On XP you could maybe use the one from Apple Quicktime, if it's installed. (or tell the user to install it if they want HTML 5 video support)
    On a Mac you should be able to use Quicktime w/o any problems.
    Consider what you are now doing: Youtube's HTML 5 video is not working with Firefox, maybe not a big issue at the moment since most people use flash anyways. But if time goes on you should ensure that html 5 videos work, otherwise u risk loosing users.
    On the other hand: Flash is also proprietary and Firefox uses the default flash installation, so I really don't see why it shouldn't when it comes to Video codecs.

    ReplyDelete
  14. Moreover, as concerning "civil disobedience" or "too small to sue", I don't think it's appropriate for us to encourage people to use things counter to their licensing in hopes that they won't get caught -- just as we don't encourage people to violate the terms of source code licenses because they will probably not get sued by the copyright holders. We have built a project and community around respect for licensing terms, and should conduct ourselves appropriately, even if we find the terms themselves distasteful.

    ReplyDelete
  15. I fully agree that Mozilla should stay away from H.264, it's simply not worth it to invest energy and/or money into this.
    However, I don't understand why video rendering of non-built-in formats is not simply deferred to the OS. For example, as an OS X user, my H.264 license is already paid for and is already part of the video codecs that ship with the machine. Why not give users the option of using whatever codecs happen to be installed on their systems? Where is the harm in that?

    ReplyDelete
  16. I thought Chrome was open too. Can't Youtube support the codecs supported by Firefox?

    ReplyDelete
  17. This page layout makes me feel sick :/

    ReplyDelete
  18. I hate the fact that the open source world is so USA-centric. Not everyone lives in USA, i.e. software patents don't affect everyone.
    For example, I can distribute open source h.264 decoder and not worry about MPEG-LA suing me.

    ReplyDelete
  19. Hi!
    I live in jurisdiction where software patents do not exist. Why are you not giving me an option to have a firefox which can play any codec? Why should I care for some idiotic and unimportant states like USA?
    Thanks!

    ReplyDelete
  20. All I can really say is THANK YOU.
    There has got to be a time and place where we put our foot down and say no! Software patents are tearing the software community apart and for us to simply allow them because it "makes life easier" is no excuse.
    I understand that a lot of the common users don't understand nor care about software freedom but that does not make it any less important. It is up to us, those that do know to fight for it.
    Those that willingly give up software freedom for usability deserve neither.

    ReplyDelete
  21. Holy horrendous layout, Batman!

    ReplyDelete
  22. Once again, someone from Mozilla is spreading FUD.
    Claim: "Mozilla should pick up and use H.264 codecs that are already installed on the user's system. I've previously written about a variety of reasons this would be a bad idea"
    Fact: Mozilla is working on such a solution for Firefox Mobile: https://bugzilla.mozilla.org/show_bug.cgi?id=422540
    Claim: "gst-plugins-ugly (...) whose use is legally questionable"
    Fact: Using it is perfectly legal. Even in the US. "legally questionable" is only the distribution, but even that is no problem if it is distributed from a jurisdiction without software patents (VideoLAN / VLC is located in France and US citizens are able to download and use VLC for 10 or so years now without ever being prosecuted for it).
    Claim: "Users just want video to work. You Mozilla people are such idealists! Yes, that is the reason for Mozilla to exist."
    Fact: No, you are not. First you started collaborating with Google on a giant commercial data mining operation (various privacy-invading features are enabled by default) and now you are actively attacking open standards with inferior codecs.
    Unlike Theora, Dirac is actually standardized (a subset to be exact). If you had adopted Dirac instead of Theora, you would a.) promoting an actual standard, b.) be still free of American patents, and c.) have a codec that does not totally break down at HD resolutions.

    ReplyDelete
  23. Hello, I'm from the Italian Mozilla community and I wish to ask you if I can translate this enlightening post and publish it on our website.
    I think we are facing a crucial problem and I fear that we have to give people the correct information, hoping many will follow us.
    You can write me at the email address I left.
    Thanks in advance. :-)

    ReplyDelete
  24. Robert O'Callahan24 January 2010 22:09

    I apologize that my multi-column layout doesn't work well for posts of this size.
    Thanks for all the messages of support.
    glandium: there is a pretty huge practical difference between "Someone might have patents on Theora that we don't know about, and might sue" and "MPEG-LA has patents on H.264 and *will* sue".
    kL, Anonymous, xurfa: sticking to the USA might be appealing but arguing that the USA is unimportant is not a serious approach.
    Jan, Apreche, MrNightLifeLover: I pointed out right at the beginning that reducing usage of Flash is a good step. However, completely eliminating Flash is not going to happen anytime soon. If this video codec issue was the only thing stopping us from eliminating Flash, then it might be worth trading off codec freedom for plugin freedom, but we don't have to make that decision yet. Looking at it another way, Flash is a problem that we already have to live with, so it's OK to live with it a little bit longer if that helps us pursue long-term goals.
    blah: We are acutely aware of the need to run plugins out-of-process, and trunk Firefox can actually do it if you flip a pref. We are working furiously to stabilize and ship that feature in the next version of Firefox.
    Tony, Markus: Dirac is great. At some point we'll probably add Dirac support. However, at typical Web bit rates, Dirac doesn't currently perform as well as Theora. The patent situation with Dirac is also currently less clear than with Theora. We'll keep an eye on it.
    Markus: The situation on Linux/GStreamer is much less bad than Mac/Quicktime, which is in turn less bad than Windows/DirectShow. Also, Mobile Firefox is currently far less influential than desktop Firefox so it matters much less what compromises we make there. Even if we end up supporting system GStreamer codecs on desktop Firefox for Linux, that wouldn't harm our goals too much IMHO.
    MrNightLifeLover: You are incorrect. Default installs of Windows Vista do not include an H.264 codec.
    Brian: I don't know much about the encoder side. Ogg plugins for proprietary editors sound good to me, though.

    ReplyDelete
  25. Come on (non-US-) guys. Firefox has already been forked because of a copyrighted icon. It's free software afterall.
    Also the internet won't die because of a useless video-tag implementation.

    ReplyDelete
  26. It is the right path. Congratulations and I hope you succeed. Anything else would really be a disaster, even worse than the current flash status-quo (people who want to make a new browser having to pay millions of dollars of fee).

    ReplyDelete
  27. I wonder what happened to Google's acquisition of On2 for the VPx codecs? I'm sure they would use that to avoid any extra wild licensing from MPEG-LA.

    ReplyDelete
  28. Robert,
    Please don't ever give in on this issue, or allow the Mozilla Foundation to succumb to the pressure of using H.264. Why did the W3C even allow it to get this far?
    Even worse this issue has turned into Apple/Google vs Mozilla and they are going to have more money to get their way.
    The good news is that Firefox has a lot more users, and if you don't give in then hopefully Google will finally find some sense. To help aid this please donate to the Xiph.org foundation to boost the development of Theora:
    http://www.xiph.org/donate/
    Why would anyone want to buy in on H.264 when Ogg Vorbis and Theora has been giving freely for so long?

    ReplyDelete
  29. Thanks for breaking it down, because like you said, a lot of people don't get the implications of this decision, nor do they understand Mozilla's stance.
    While some of the reasons Google offers for this choice do sense, such as the worry over file size, I can't help but think that it is a needle at what is one of their biggest competitors. At the very least, if it is not an intentional jab, they are likely very pleased with themselves.

    ReplyDelete
  30. I would like to salute every one at Mozilla for doing the right thing. If it wasn't for you guys we would still be paying for good browser or using Internet Explorer at mercy of Microsoft and malicious stuff on Internet.
    I cannot even imagine how bad the situation would have been on operating systems other than Windows.
    Thank you for doing the right thing.
    - A user of Opera browser who is grateful for being liberated by the phoenix.
    p.s. to Google: I WANT MY WEB FREE, kindly do the right thing and dump support for proprietary stuff that breaks Internet. Please remember
    "Don't be evil".

    ReplyDelete
  31. Thank you for this. I think Mozilla's stance is correct and I am disappointed that these websites, YouTube and Vimeo have had the opportunity to advance improvements in freedom and have not taken the leap.

    ReplyDelete
  32. Thank you for the clear information presented and the follow up comments.
    The legal and philosophical approaches are usually more subtle than the "it works!" argument and greatly appreciated.

    ReplyDelete
  33. Petition to get ogg/theora on Youtube. Please sign
    http://www.petitionspot.com/petitions/oggandyoutube/

    ReplyDelete
  34. Stay strong Mozilla, keep fighting the good fight. Too many people are willing to ignore the injustice of software patents and unreasonable restrictions just to watch a few videos on Youtube; thankfully we have institutions like Mozilla willing to stand their ground.

    ReplyDelete
  35. I already posted this comment on gandalf's blog, but in Polish, so I'm going to repostt it here in English. :)
    We, the Mozilla community, need to make Ogg Theora not suck. Just like we made web browsing not suck (let's face the sad facts - people started using Firefox because it was better, not because it was morally superior - and we didn't even talk much about this superiority either for a long time - the "mission" tab on Firefox homepage appeared only recently with 3.6).
    Current solutions for Theora playback suck:
    a) "use Theora only, and use Cortado Java applet to play video in IE and Safari" - sucks. Java applets are not only an obsolete technology, they are such a nightmare - because of their unacceptable performance and start-up time, and due to a lower user base than e.g. Flash. No way anyone can talk their boss into supporting Ogg Theora through a fracking Java plugin for 30-70% of the user base, when Flash solutions are better and, if your company is big enough, you can always throw some money on a h.26x license.
    b) "let's have two copies of every video file, one in Theora, the other in h.26x, and use a Flash player for IE" - sucks. First - doubling the size of a few video files is not a problem, doubling the size of thousands of movies is a great problem. And, if there's a Flash plugin for h.26x, talking management into having another copy in Ogg, when the Flash plugin can handle the h.26x is - business-wise - nonsense.
    So, what needs to be done?
    We need two things.
    1. A good BSD-licensed Flash-based player for Ogg Theora/Vorbis content, so that any company can steal it for their own use. So that they can send Theora to Firefox, Opera, Chrome and to the Flash-based player under Safari and IE.
    2. An even better BSD-licensed streaming server for Theora. I'm not sure whether IceCast is good enough, though, and how it compares to proprietary h.26x streamers.
    If we have a BSD-licensed Flash-based Ogg player and a great BSD-licensed streaming server, I can easily imagine e.g. my employee (an Alexa Top 10 site in Poland) switching happily to Ogg, due to all the problems around WMV, FLV, h.26x etc. I can't imagine myself talking my employee into (a) nor (b) above, though, unless Firefox doubles it current 50% marketshare in Poland, which I don't think is gonna happen, even though I'd love it to.
    We've got to do this. We can't sit back and wait, because everyone will just use h.26x in three years.

    ReplyDelete
  36. Since 2004, Mozilla has had baked-in support for automatically installing Flash on the first encounter if the plugin is not found. A nice little yellow infobar pops down (a brilliant UI innovation), prompting you to install it with a few clicks, even without root access on both Windows and Linux.
    You've also recently implemented automatic update checking for Flash. Since it's your biggest security hole, you throw up a big nasty "update now" warning on launch if the user has a known-vulnerable version.
    Mozilla even initially distributed the Flash binaries under license themselves via addons.mozilla.org — is that still going on?
    Flash is shitty, nonredistributable, closed-source, restricted-platform, proprietary, and subject to even more patents but you're willing to go to great lengths to help your users use it. Why not do the same thing for ffmpeg, which is merely patented?

    ReplyDelete
  37. google plan:
    1. release chrome with encumbered video support.
    2. support that format on youtube.
    3. turn off the fall back. everyone move to chrome
    4. when authors realized the legal hassle after 2017, release googlevideocodec, already working on chrome and youtube.
    5. get some fat fees from content creators to use their encoder.

    ReplyDelete
  38. use openBSD's solution.
    buy a server rack in Canada.
    distribute all the software you want from there.
    worried about your developers? only serve encrypted source code via ssh+svn for example. that way they can upload code from their oligarchical country, such as the US.

    ReplyDelete
  39. Youtube decides to only go with H.264, and 95% of Firefox users will on that same day switch to Chrome, Safari or whatever browser that DOES support youtube at that moment. Don't underestimate the amount of people that just want a browser that works for their standard internet needs, and couldn't care less about WHY "firefox won't support youtube".
    Big help yer shiney ideals are going to be at that time...

    ReplyDelete
  40. Zebastian Hedlund25 January 2010 03:31

    I solved it, I installed chrome!

    ReplyDelete
  41. Ok, I can accept that you cant ship H.264 in the USA.
    It isn't a problem elsewhere. Let it work in versions for other countries.
    Is there a legal reason you cant do this? If there isn't, then you are being extremely rude to the rest of the world by attempting to force the effect of US laws on them.

    ReplyDelete
  42. Some of the comments here do have a point, but I do understand the contradictions. The ends do sometimes justify the means. Even though it is a contradiction to rely on the flash fallback, Flash has been a reality for many years, and the flash install base is very large. Relying on the flash fallback is just maintaining the current state, while pushing towards the ideal outcome of using open video codecs. I am sure that google is currently working on ogg versions of youtube videos.

    ReplyDelete
  43. About the proposal that localized releases of Firefox include h.264: The intention may be to stick-it-to-the-patent-holders, but it's important to consider that this idea punishes the users far more than the suits.
    Where software patents do not apply, the patent holders lose nothing; where software patents apply, the patent holders lose nothing, but "because you live in the US," many people who use Firefox are excluded from a potentially important feature.
    roc: thanks for posting this breakdown. :)

    ReplyDelete
  44. Pseudonymous Coward25 January 2010 07:10

    Thank you, thank you so much, Mozilla, for standing by the ideals of free software.

    ReplyDelete
  45. abhishek manocha25 January 2010 07:39

    Your stand as pointed by many is contradictory on many fronts.

    ReplyDelete
  46. The amount of misinformation in these comments is ridiculously sad.
    H.264 is not royalty-free, meaning anyone implementing/shipping a encoder/decoder has to license (pay) heavily to the MPEG-LA. This is a huge problem for interoperability and free access to content, because then one would be locking into a single non-free codec.
    What if each browser had to pay royalty fees to the W3C to implement HTML ? (In fact they already pay a membership fee but that is different)
    Or in another case, look at the current de-facto monopoly that Ms Office has. People exchange MS Office documents around, out or sher ignorance, instead of using open format documents, like pdf or odf, which forces everyone to have a MS Office available, meaning that people need to **pay** Microsoft a windows and office license.
    This is not how freedom is defended.
    I'm personally quite glad that Mozilla at least stood against h.264 with Opera, else Mozilla or Opera would be forced to license h.264 and make the MPEG-LA a huge profit, while locking other royalty free codecs out of the market. Google licensed h.264 which makes me believe that it was a unilateral management decision, and not something discussed by the big engineering mass they have.

    ReplyDelete
  47. Releasing localised versions of Firefox with H264 support wouldn't hurt US users in practice, because of course such versions would leak into the US, and at the worst Americans would have to put up with the British spelling of "Colour" in the browser options. It would be a patent violation, but it would be the user's violation and not Mozilla's, and I can't see much likelihood of prosecution.

    ReplyDelete
  48. Robert O'Callahan25 January 2010 09:46

    Markus: regarding "privacy-invading features are enabled by default", I have no idea what you're talking about, because we do no such thing. For example, the phishing-site detector uses a protocol that is very carefully engineered to conceal which sites the browser is actually visiting. If you're going to flame Mozilla on an unrelated topic to the post, at least please make your accusations specific enough to be refuted.
    Also, do you have a legal opinion that says you definitely can't be sued for using gst-plugins-ugly in the USA? On the face of it, it seems to me MPEG-LA could successfully sue you if they chose to.
    marcoos: I doubt you can make a Flash-based Theora player (in Actionscript, you mean?) with acceptable performance. There are Theora players for DirectShow and Quicktime, perhaps users could install those? I dunno.
    I don't really know anything about the streaming side. Our friends have Xiph have more to say about that. Silvia Pfieffer had a tutorial about this at LCA: http://www.lca2010.org.nz/programme/schedule/view_talk/50180?day=thursday
    Please stay in touch with Gandalf about this issue. If more sites in high-Firefox-share countries switched to Theora, that would really help us.
    Fred Blasdel: I commented above why Flash is, in some ways, a lesser evil because it's an evil we already have to live with to be a viable browser. Given we have to live with it, we have to make the best of the situation by making sure it's up to date etc. Furthermore, there is also a pretty clear understanding on most sides that Flash (and plugins in general) are not part of the open Web platform that we are striving for. All browser vendors, except possibly Microsoft, agree that functionality needed by general Web apps should be available without resorting to proprietary plugins. The same cannot be said for encumbered codecs.
    gabriel: I don't think OpenBSD's solution would work for us. I don't think MPEG-LA's lawyers are going to stopped by a simple workaround like that. A judge is going to see straight through it. And of course, even if such a legal hack (or any of the other legal hacks people have suggested) actually worked, it still leaves content providers and other browser vendors (current and future) at the mercy of the MPEG-LA.
    Ian: I don't see any major site disabling Flash fallback anytime soon. If it happens, and we bleed serious market share, we certainly would have to reevaluate whether the damage is worth it. Obviously if we lose all our market share our influence vanishes and that would not be the best way to serve our mission.
    Ben: thanks for the feedback. As you note, these are all things we're working on. I think we'll actually be able to support accelerated video rendering on a lot more platforms than Windows 7 and D3D10.
    Matthew: How do you propose we limit functionality based on the country you're in? Also, as I noted above, if we hack around the license issue somehow for some subset of users (or even all users), we'd still leave Web authors high and dry.
    mors: thanks for the support. I want to point out that paying W3C membership fees is not required in order to implement a browser, or any W3C specs. If it was, we'd kick up a huge fuss! W3C membership is basically just a donation ... membership may be needed to be a formal member of some of the spec working groups, which is unfortunate, but not too serious.

    ReplyDelete
  49. > marcoos: I doubt you can make a Flash-based
    > Theora player (in Actionscript, you mean?) with
    > acceptable performance.
    actually it is possible to embed C libraries in flash applications. I think the technology is called "Alchemy", see for instance:
    http://blog.debit.nl/?p=67
    Potentially libtheora could be embedded there?
    I also think that doing this would have a huge impact, probably much bigger than anything else that could be done at this point.
    One thing I was wrong on previously, technically required H.264 doesn't mean making a browser means paying fees to MPEG-LA, you can get away by relying on the plugins on the computer, though this is still not OK.

    ReplyDelete
  50. Robert O'Callahan25 January 2010 10:32

    Alchemy still forces your C code to run on the Flash VM. I'd still be surprised if the performance is acceptable. Someone should try it, sure.

    ReplyDelete
  51. yes, thinking about that alchemy thing... it could probably decode theora but then there's the HW-accelerated rendering problem... it would probably become way too complicated.
    In then end probably the only solution is a completely new browser plugin that is not offering a VM like flash but that would know to display Theora video just enough to make such a player, and nothing else. It could be almost the same code as firefox's and code just split out and relying on a different html syntax -based.
    two problems:
    1. that plugin doesn't exit
    2. that plugin has 0% market share
    i guess it wouldn't help much, youtube and others would say that the hurdle to install the plugin would turn away users and it's not worth it.

    ReplyDelete
  52. ah, such a plugin would need to at least interact with JS to enable youtube and others to customize its display... which makes it suddendly a huge task.
    i guess if there was an easy solution, it would already have been done :-(

    ReplyDelete
  53. Love live, mozilla ..

    ReplyDelete
  54. ****Mozilla should pick up and use H.264 codecs that are already installed on the user's system.
    That is the solution! It is not up to you to tell users what codec they should/should not use. Users will make the choice themselves and having the ability to use an existing system codec will open up html video to all other codecs. Please have some sense.

    ReplyDelete
  55. What Theora really needs now to spread the world is "native" support from Adobe in Flash. Such a step would make Theora equally accessible to H.264 and video servers like YouTube or Vimeo could switch. The question is how to make Adobe acquiring open video standard. Perhaps making Theora better could work.

    ReplyDelete
  56. >Posted by: glandium at January 23, 2010 11:52 PM
    >I live in a country free from software patents and I don't want to suffer from another country's legislative problems.
    Yeah, really? And where is it? Because 100% it isn't in France¹:
    FR 564,597 
    FR 630,157 
    FR 2,599,5773 
    FR 1,467,491 
    FR 1,487,113 
    FR 1,550,219 
    FR 0460751 
    FR 0443676 
    FR 1,359,769 
    FR 1,383,338 
    FR 1,406,453 
    FR 1,359,767 
    FR 1,135,934 
    FR 1,468,566 
    FR 1,468,567 
    FR 0909099 
    FR 1,401,213 
    FR 1,742,482 
    FR 1,742,483 
    FR 1,742,484 
    FR 1,387,585 
    FR 1,406,451 
    FR 1,424,856 
    FR 1,450,565 
    FR 1,452,039 
    FR 1,478,188 
    FR 1,499,133 
    FR 1,515,567 
    FR 1,596,608 
    FR 1,739,972 
    FR 1,739,973 
    FR 1,742,479 
    FR 1,742,480 
    FR 0279053† 
    FR 580,454 
    FR 1,246,131 
    FR 1,510,078 
    FR 1,596,604 
    FR 573,665 
    FR 598,904 
    FR 898,426 
    FR 907,289 
    1 - http://www.mpegla.com/main/programs/AVC/Pages/PatentList.aspx

    ReplyDelete
  57. Of all the arrogance in the world. The world still don't evolve around America. The majority of countries don't have software pattens. Why the hell should we care if h.264 might be a problem for some. This is the U.S encryption laws all over again. If I want to use h.264 on my system that's my right. FF should only pass the request down to my codec of choice.

    ReplyDelete
  58. As someone who doesn’t live in the US, shipping the (US-)patent-encumbered versions only outside the US, and providing an H.264-stripped version inside the US, would be a good way to make a point, if nothing else.
    It’s similar to the situation with encryption back in the 1990s (or GIF images last decade).
    I can see how that might be construed as giving aid and succour to H.264 though.
    Given Mozilla's self-imposed constraints, the notion of making Theora (and Dirac) more palatable by encouraging Flash support for them sounds the most promising.

    ReplyDelete
  59. What's wrong with a plugin? Let the plugin handle completely and be done with it.
    Yes, there will be some interaction with the rest of the website, but it can't be that difficult. The DOM will be set in HTML 5 ...

    ReplyDelete
  60. Johan Krüger-Haglert25 January 2010 18:50

    Jonathan Feinberg:
    Speaking for myself all I can say simply because I didn't knew about Dirac earlier, thanks for the info.
    I assume it may also be because ogg theora is more wellknown by name and maybe some people think people already have support for it but I doubt that. Small format noone use.
    So well, yes, I would prefer Dirac over Theora since it seem to offer better quality.

    ReplyDelete
  61. Great work! This is an inspirational read, and I hope you guys are able to turn some heads about this codec issue.
    What can we humble users do?

    ReplyDelete
  62. And about Dirac codec (for SD and HD videos) and Schrodinger lib?
    Dirac is perfect for this job! You should promote it!

    ReplyDelete
  63. Robert O'Callahan25 January 2010 20:44

    Johan, ludo1243: It seems you guys didn't read my earlier comment. Dirac is great at high bitrates --- for "archival video", for which it was designed --- but at typical Web bitrates, which are much lower, *it is not as good as Theora*, currently.

    ReplyDelete
  64. "Markus: regarding "privacy-invading features are enabled by default", I have no idea what you're talking about, because we do no such thing."
    Google is the biggest data miner on the planet.
    You send every character typed into the search bar to Google, because the "suggest" feature is enabled by default. Granted, after many users protested, Google now claims to store that data not as long as regular search data, but it's still stored.
    Why is Google the default search engine anyway? You claimed in your blog post that Mozilla is an idealist's organization. If you were idealists, Forestle http://us.forestle.org/en/ would be the default, but your idealism is lead purely by money. You use Google, because Google pays most and you refuse to use AVC because some of your money may be needed to spend on licensing fees.
    You have no problem helping Adobe install proprietary but costless plugins like Flash.

    ReplyDelete
  65. This is mentioned in other comments, but you think Ogg is any better on the patent front? I'm interested in your take on this.

    ReplyDelete
  66. Robert O'Callahan25 January 2010 21:30

    OK, so your problem is that we send characters typed into the search box, which has a Google logo next to it, to Google. Clicking on that logo lets you choose to send searches elsewhere with just one more click. That's not easier in any other browser, and in many (e.g. Safari and Epiphany) you can't change the search engine at all without editing config files or code. We provide more than enough choice and disclosure here, and more than any other browser.
    Note that Epiphany doesn't get any money from Google; they use Google because that's what users want. For the same reason we made Google the default in Firefox before Google was giving us any money.
    (I don't know what powers Forestle, but a quick test with a search for "mozilla pymake" shows both Google and Bing bringing up the pymake documentation page as the first hit, but it's not anywhere in the first page of Forestle's hits.)
    Similar issues were thrashed out in this LWN thread. The comments there are worth reading.
    http://lwn.net/Articles/346393/

    ReplyDelete
  67. Robert O'Callahan25 January 2010 21:34

    Chris Latko: I replied to that in one of my comments above. There is a pretty huge practical difference between "Someone might have patents on Theora that we don't know about, and might sue" and "MPEG-LA has patents on H.264 and *will* sue". So yes, the patent situation with Ogg Theora is obviously better.

    ReplyDelete
  68. Interesting to see so many people (still) thinking that they live in a software patent-free jurisdiction. That is highly unlikely and if you live in Europe at least, the major difference is that the EPO et al will usually 'lie' about this issue (depending on the audience) whereas the USPTO et al will not.
    [I put “lie” in scare quotes there because they generally don't actually lie outright - they'll typically exploit legal technical subtleties and say logically strictly true but utterly disingenuous things such as “we don't grant patents for software programs”, and “only technical inventions such as might be found in dishwashers and car braking systems are patentable”.]

    ReplyDelete
  69. The more this comes up the crankier it makes me. Right now I am very cranky indeed, so please forgive this comment directed at a few commenters and probably a lot of non-commenting readers:
    *"The internet is not here to be your [INSERT VOCIFEROUS PROFANITY HERE] TV!"*
    The issue here isn't promoting "easy consumption", or even "sustainable" or "healthy" consumption (thought that's a factor). The issue here is to promote *participation*, which is the aspect of the internet most endangered by the need to bribe one's way through various proprietary licensing tollbooths before being allowed to contribute.
    Apple, Microsoft, and even Google are trying to choke off legally-free participation precisely because *they* are running the tollbooths. I've read that Apple, for example, controls some of the H.264 patents.
    The only way to keep the internet open to participation is to have at least some readily-accesible roads without tollbooths. *That's* what this is about.
    Sure, these metaphorical roads don't get well-maintained until they get enough use, which in this context means having a substantial amount of readily-available "content" in the legally-free formats. That's presumably why everyone's obsessed with YouTube™ - if they made even a substantial part of their non-HD video available it'd represent a huge jump in attention. I don't think Google wants a large free-access road popping up next to whatever tollbooths they're working on right now, though, so they seem to be carefully "harmonizing" their public statements, responses to polls and feature requests, and so forth to make it seem as though Ogg Theora (and Vorbis, don't forget) don't exist, while striving to give the appearance of support for "open HTML5 web standards".
    Given how central the goal of facilitating participation has been to the philosophy behind Mozilla Firefox's development, I'm slightly baffled why anyone with an "I don't care about *participation*, I just wanna *consume*" folks even care what Mozilla does with Firefox <video> and <audio> - just go buy a standard Mac or Microsoft-Windows/Internet Explorer system, or perhaps an iPhone®. "Consuming" is pretty much what those are for. (There's bound to be a "Bread and Circuses" app or two in the "app store".)
    I know that sounds facetious, but I'm serious. With the possible exception of Richard Stallman, I don't think most of us mind that "consumer" toll roads exist, so long as they aren't actively choking off our ability to build and use toll-free roads as well.
    That said - "Big Content™" doesn't want tollbooth-free roads around, so I'm cynical that any amount of pressure from small participants on the internet will have any real effect. I think efforts might be better focussed on making it easier to create and find independent material in legally-free formats. As an aside, supporters may want to show some love and attention to <audio>, too - it's a lot simpler for a hobbyist to produce audio-only content than to deal with full video production. Since Steve Jobs will no doubt never allow "Oggcasts" in the iTunes® store, some alternative means of distributing and finding Ogg content will obviously be needed...

    ReplyDelete
  70. The thing that I don't see is why, given everything you've said, support gstreamer. It seems your answer above is that mobile doesn't matter much. But I'm surprised it is needed just for mobile; IMO there are lots of things Fennec needs more than a very basic HTML5 video support. Is there a lower barrier for getting a new video backend in the main source tree if it is just for Firefox mobile? (And why won't distros use it once it's checked in and a Firefox (mobile) has shipped with it - it's been declared good enough to be called Firefox).

    ReplyDelete
  71. Very well written summary to address the often-heard concerns about using open video. Thank you for spreading the message about using open video, because when people really understand all the limitations of H.264, then they do come around to embracing free (as in freedom!) video technology.

    ReplyDelete
  72. Great article btw - It seems to me that the MPEG-LA have to be more reasonable with their licensing. By that I mean that to get any reasonable performance with H.264 you need hardware acceleration. The owner of any hw supporting this has already paid necessary licensing for this - this is no different to licensing for the HDD, LCD, Processor, Wifi and everything else you can think of in your computer. Now given that regardless of the sw stack you use on your PC you have already paid some money to MPEG-LA, it should be OK for a free software stack to use the hw features, presumably with upper layers of the codec still being handled in software.
    It would require MPEG-LA to change the royalty distribution such that the patent holders of parts not implemented in hw were compensated to license their patents in a compatible way with the GPLv3, through revenue obtained from licensing the hw IP or chips.

    ReplyDelete
  73. One more thought: the time to head this battle off was when WHATWG removed the requirement for Ogg because of Apple and Nokia (the first time they did it a couple years ago, not the debate about removing the comment in the spec this summer). Since you can't mandate H.264 (patent policy reasons) that should have been a red flag. That debate was about nothing or Ogg, with nothing defaulting to H.264. Now there is a much steeper climb, and marketshare and ideology is what you have going for you. I don't see what the next step is once Microsoft ships IE9 with H.264, but that's why they pay you to think about this rather than me.

    ReplyDelete
  74. I found it pretty amusing to find out that Youtube's HTML5 beta doesn't even work in *Google's own Chromium browser*.
    Youtube can keep its broken H.264. In return, I'll keep my grab-url-and-use-mplayer method of watching which bypasses all their advertising and Flash.

    ReplyDelete
  75. so as a web developer I really want to be able to embed video without flash. you guys are making this difficult. oddly I found this blog by searching for petition mozilla h.264 - please don't ingnore your users or they will eventually ignore you.

    ReplyDelete
  76. Why not use your public exposure to create an easy, fast way for users to print-out, address and send "template style" physical letters to appropriate departments in the usa to complain about them not forcing patent owners to "disclose or lose" any patents which relate to any visible use by the public? -ie: no more submarine patents, -problem solved for uncertainty about free patents! - and we just keep this up untill something "gives".

    ReplyDelete
  77. Like others here, I find it highly amusing that Mozilla continues to go to such great lengths to protect and promote Adobe Flash, which is horrifyingly closed and proprietary to a single vendor. Yet they turn their noses up at H.264, which is not dependent upon any one vendor. Payback for all the Tamarin code Adobe "donated" to you guys a while back?
    Stop being such laughable hypocrites on this issue.

    ReplyDelete
  78. @ant: Chrome and Chromium have a rather bizarre relationship with HTML5 Video (and audio). It would turn out that the media backbone (compiling ffmpeg) that they use for it happens to have have after all other format's save for the OGG family commented out of the code.
    Simply put for every build of Chromium, H.264 and MP3/MP4 playback is disabled. And from what Google has said about problem, it's nothing that they can fix... it comes down to MPEG-LA licensing and getting a license from MPEG-LA for your own project. It not something that Google can give to the downstream. And the comments in the code itself more or less spell out 'If you un-comment these flags, without obtaining the proper license, you do so at your own peril.'
    I think that it's rather telling that Google, a rather large company, had to run out an BUY a license from MPEG-LA just to have H.264 and MP3/MP4 playback for Chrome.

    ReplyDelete
  79. Just wanted to say thank you for trying to keep the web open. Are there any specific patent issues around Dirac you are looking into?

    ReplyDelete
  80. Robert O'Callahan26 January 2010 12:45

    As far as I know, we haven't done any patent analysis of Dirac yet.

    ReplyDelete
  81. Actually you should delegate all the video codecs to the underlying OS.
    Then maybe pop up a window to say "Codec xyz is not free that means blah blah...", but still the tag doesn't specify the coded, but just video, as doesn't specifiy if jpg or png or whatever.
    The same should apply for videos.
    Cheers,

    ReplyDelete
  82. >The H.264 patents will expire soon, and then we'll be OK. Many H.264 patents don't expire until 2017. Anyway, H.264 isn't the last word in video compression. There will be an H.265 and the same set of problems will persist.
    So if h265 is so much better that major players will want to upgrade to it, won't that leave Theora even further behind than it seems to be already? Isn't it too late to talk about "replacing" h264 in html5, shouldn't the talk be about, what can you come up with that's better than h265 that is free and people want to upgrade to?

    ReplyDelete
  83. Robert O'Callahan26 January 2010 14:11

    Theora definitely isn't the last word in unencumbered video codecs, either. It's just the best thing we have right at this moment.
    Generally I don't like to talk about future work until that work has produced some actual results. Anything else is just empty words.

    ReplyDelete
  84. >The H.264 patents will expire soon, and then we'll be OK. Many H.264 patents don't expire until 2017.
    The expiration date is more like 2028 in the US.
    http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-July/020737.html

    ReplyDelete
  85. *** RE: Mozilla should pick up and use H.264 codecs that are already installed on the user's system.
    After reading both this article, as well as the other linked post (which really only touched on DirectShow), I still believe you're wrong. As much as I think that promoting software freedom is an important role, doing so at the expense of expected end-user functionality is wrong. But instead of just disagreeing with the points for not using a system-provided codec, lets instead look at the benefits of it.
    1. Additional video codec's. Ya - the obvious one, right now its h.264 that all the users want, but later it'll be h.265, or something else, who knows. The point being I don't want to have to wait for a new version of my browser to be released just because a new video codec is being used somewhere. Codec's need to be able to be added/removed from the system without touching the browser.
    2. Codec optimizations. Using h.264 as an example - lets say we're watching some 1080p HD streaming content, thats going to need a lot of CPU power to decode, unless the client is on a system with a hardware offload of some type. Do you plan to add hardware acceleration to firefox for every codec and every chip out there? DXVA on windows, VDPAU / VA-API / XvBA on linux, Broadcom decoder chips in embedded platforms, etc. This is probably a big part of the reason FireFox mobile would go with a system-provided codec - on a mobile device you'll never have the CPU power to decode without a device-specific module for its decoding hardware.
    3. Promoting software freedom. Yes - I'm using that argument for the other side. Remember all those millions of XP users that don't have a h.264 codec. You aren't helping them either way if you use a limited built-in video support, you're pushing them back to IE. Instead let their system pop up telling them they don't have the software required for playback. Let them deal with paying for a license for a playback codec if thats what they need to do (and those of us in countries without stupid laws can skip the paying part). And let them see that if the content they were after had been provided in a free format they wouldn't have had to deal with that hassle - they can start petitioning their content providers for the format they want then.
    4. Code re-use. I like the wording you used in the other article - "But if we're going to ship our own video infrastructure ...". That sounds like a lot of extra work, developing, testing, shipping, supporting, etc. an entire cross-platform video infrastructure. It sure sounds a lot easier to me to re-use the video infrastructure that already exists on all those systems, and only have to develop/test/support an interface/wrapper for each one.
    In the end, this feels to me very much against end-user freedoms. I don't like software that tells me what do to or how to do it - its not my browsers role to tell me what video codec I should encode my home-video's in. If I choose to use a patent-encumbered format, then thats my choice, and I expect my browser of choice to play that file still.

    ReplyDelete
  86. > It pushes the software freedom issues from
    > the browser (where we have leverage to
    > possibly change the codec situation) to the
    > platform (where there is no such leverage).
    It is just bad engineering, and rude, to *intentionally* not use a modular architecture for your implementation (like GStreamer) which allows for integration with the facilities provided by the host system, as well as pluggable support for alternate or future formats.
    If you really want to stand behind this argument, then you should also drop support for the evil and proprietary platforms like Windows and MacOSX altogether. How many Mozilla resources are wasted on cross platform support for these closed systems? I run GNU/Linux (for Freedom) on my desktop, so that suits me fine.

    ReplyDelete
  87. Robert O'Callahan27 January 2010 00:40

    We have a modular architecture. It doesn't support integration with the host system codecs at the moment. It could --- in fact, we dabbled with some prototypes that did --- we just choose not to invest there.
    If your comment is really "why aren't you idiots using GStreamer?", then that's an interesting technical discussion to have, and it probably deserves a blog post of its own. But I wouldn't be surprised if we do end up using GStreamer at some point in the future. (I wouldn't be surprised if we don't, too.)
    If we could drop support for Windows or Mac without losing much influence, we probably would. But it's important to remember that the only reason you can browse the Web without IE today is because we shipped Firefox to tens of millions of Windows users.

    ReplyDelete
  88. It feels like the original intent of the and tags are, once again, being hijacked by corporate interests who have bullied enough people to force Ogg out of the standard.
    For those that insist Firefox pick up a plugin to play H.264, why not the other way around??? Since H.264 is the most encumbered codec and Ogg is free to redistribute (that's the biggest point, people!!!), why don't the other browsers support Ogg natively and pick up H.264 in a plugin? What's good for the goose is good for the gander, right?
    Honestly, I thought the original intent of these tags were to allow multimedia playback without the need for additional libraries, players, or plugins. With that in mind, it baffles me why some folks want Mozilla to give in and others think it's hypocritical of Mozilla to be OK with Flash...for now. Flash won't be built into Firefox. Firefox isn't redistributed with Flash built in (and no other browser that I know of does so, either). Again, the whole issue was initially about being able to play multimedia without plugins or external players. Flash, in this context, is considered a fallback, not endorsed as a solution.
    Therefore, what can Mozilla distribute inside of Firefox without being tied to a known-encumbered codec and without a plugin (yet still offer that possibility later)? H.264? Is that codec really and truly redistributable with no known legal restriction? No? There's your answer. Instead of trying to find workarounds for this sticky mess in trying to force H.264 down the throats of those that simply cannot afford it, why not go with a codec that is free of such encumbrance for everyone?

    ReplyDelete
  89. I think that the fact that Google only enabled h.264 HTML5 video on youtube has more to do with the fact that all their videos were already encoded in that format (at 3 different resolutions), for iPhone and Android support. Therefore, it was relatively easy to just turn on the switch for beta HTML5 embedding.
    Transcoding all those videos to Ogg Theora (with multiple copies for SD, HQ and HD) would require a major computing effort and storage space availability, that, sadly, just isn’t worth it at this point. Remember, it took MONTHS in 2007 for youtube to transcode all of their h.263 FLV videos to h.264 mp4's for iPhone support. And that was before Youtube added 720p and 1080p HD video support. They'd literally have to double their datacenters' storage space!!
    Also, Apple should be blamed for being the only browser vendor refusing to feature Theora support for its video tag support.

    ReplyDelete
  90. I think the reason why youtube and vimeo use non-free codec is because there is a problem of copyright in the video which is hosted on those sites.
    May be Mozilla should make a plugin one like flash so it would be possible to play these video on firefox. If you think about it, chrome and safari are both nonfree browsers (chrome is open sourced as chromium but chrome has few functionality which doesn't exist in chromium)

    ReplyDelete
  91. Re: Joshua Cogliati
    "Mozilla should pick up and use H.264 codecs that are already installed on the user's system."
    I'm with you Josh. I'm a heavy Mac user, and I personally would prefer h.264 over the ogg/vorbis argument. (ducks for the tossed tomatoes)
    I do love my Firefox, however I would just like Firefox to allow the video to pass through even when I'm on my Win7 machines. I always have quicktime installed, so codecs are not a problem.
    Firefox should not be making the judgement call on what we choose to view on the web.
    There will always be some sites the push out h.264 and some may push out ogg in the future.
    However, the Mozilla community is keeping us in the dark (I'm a US user) by forcing us to have one over the other.
    Personally, I would like to have my cake and eat it too.
    Anyone with me?

    ReplyDelete
  92. "[2] RANT: in many cases, free implementations of heavily patent-encumbered technology are harmful to the free software ecosystem"
    I want to disagree here. Without free implementations of MPEG2/4, I would not be able to play videos on my desktop. Not even per YouTube download tricks, simply nothing. It would be a severe problem for the free desktop in general, because fun is important.
    Also, without free implementations of MPEG2/4, I would not be able to use MythTV, nor be able to write my own alternative to it. I would still be slave to TV schedules. (TiVo and similar never reached Europe.) This is an area where open source can really innovate and we're almost a whole decade ahead. Without MPEG, no TV (DVB), no DVD, nothing.

    ReplyDelete
  93. What do you see as the outcome of all this mess, 5 years down the road? Is Mozilla capable of exerting enough pressure?

    ReplyDelete
  94. What is your position on x264:
    http://en.wikipedia.org/wiki/X264
    ?
    "... a free software library for encoding video streams into the H.264/MPEG-4 AVC format. ... released under the GNU General Public License."

    ReplyDelete
  95. Robert O'Callahan30 January 2010 00:02

    pd: x264 sounds like great technology. Overall it's probably a good thing, since you can use it freely in many jurisdictions. I get annoyed by x264 advocates who push the arguments I refuted in this post, though. In the USA and probably in Europe, your freedom to use and redistribute x264 (without fear of being sued) is curtailed by patents, and x264 advocates are in denial about it.
    Matthew: I have no idea. The only thing I'm sure of is that in the next few years there will be some large unexpected events that will invalidate any predictions I could make now :-). Our job is simply to keep pointing in the right direction, do our best, and be ready to seize any unexpected opportunities that come along.

    ReplyDelete
  96. First, I applaud Mozilla for taking this stand for all of us. Please don't give in. You are finally of the size where your decisions to or not to support proprietary solutions matter.
    Second, I'm very disappointed with Google/YouTube for defaulting to a proprietary codec. Very disappointed. They could have made the decision towards open codecs "done."

    ReplyDelete
  97. FWIW, this post has helped me greatly with convincing other people at Fronteers (the Dutch organization for front-end web developers) to publish the videos of our recent conference solely in Theora; thanks, and please keep writing about these things, your posts are invaluable!
    (I also wrote a post about our decision: http://fronteers.nl/about/theora-choice )

    ReplyDelete
  98. Politics and Programming Do Not Mix.

    ReplyDelete
  99. There seem to be some misinderstandments in the comments about some things:
    x264 is an encoder, it is NOT able to DECODE video!
    You can only use it to encode video to h264, not play it. There for it's useless anyway, even without patents.
    About dirac, Dirac is actually a format.
    But dirac-research and schrodinger are IMPLEMENTATIONS(=codec) of that format.
    While they aren't optimized for low-bitrate web-oriented video. What's stopping you to have a look at the possibilities to improve these codec to have low-bitrate optimizations?
    And Mozilla, for pointing to the issue of codecs.
    You should build a short text with a link to a more complete explanation of this issue that comes up every time the user tries to play a h264 video. This way average joe understands the reasons and knows that something called MPEG-LA is responsible for his uncomfortability. This ain't the blame game. This is letting users know why it doesn't work. You really should do something like this. It would inform users about the problems and maybe they will ask more for ogg theora video's. There is nothing stopping you from saying MPEG-LA is asking unreasonable licence fees for everyone.

    ReplyDelete
  100. Please support aside the ogg container also the mkv CONTAINER.
    It's being used a lot.
    e.g. HandBrake transcodes theora and vorbis in that format.
    It's a very nice Open Source multimedia conTAINER.

    ReplyDelete
  101. As an American myself, I actually do like the idea of non-US localized Firefox including H264. Whats the point of all the protests against EU software patents if Europe lives under the yoke of US patent law?
    And what a perfect demonstration of the harm software patents do by sending crippled software to the US. It would be a real learning experience for millions of Firefox users. It would be a stain on our national geek pride. :)
    ...assuming that's legally possible. I've heard it claim that if you distribute to the US, then you have to pay patent royalties even on stuff you distribute in Europe. But IANAL and this was hearsay from a NAL. But I do think this technique should be seriously thought about.

    ReplyDelete
  102. I agree with other posters who have noted "the Flash fallback works" commentary isn't really true in many cases.
    In practice Flash is horribly bug ridden on Mac OS/Linux/BSD/Solaris and cripplingly slow on slower computers running Windows (older computers, or netbooks). So much so that Apple intentionally ship an older (more stable) version of Flash with Mac OS 10.6, and of course continue to refuse to allow an implementation on the iPhone OS (a situation which I am personally quite happy with).
    Perhaps more importantly I don't think it's fair to hold that it's an ideologically a better option to force users to depend on Flash for embedded video. I'm a lot more concerned about how proprietary and troublesome the widespread use of Flash is than about the licence behind a specific video code (which is not to say that it's not a valid concern, but I do think it's the lesser of two evils).
    The net effect of refusing to budge from this position of resistance will be to stifle the roll out of HTML 5 video, to bolster the case for continued use of Flash and to provide another reason for people to choose a browser like Google Chrome over Firefox.
    The position outlined above on H.264 seems idiosyncratic at best, but hey it's ultimately up to the Mozilla developers. I would dare suggest though, unless they are determined to stick to the decision to not implement it till the bitter end, it would be less damaging for the perception of Firefox to do it sooner rather than later.

    ReplyDelete
  103. Robert
    I came across this article quite by accident. Kudos for raising the issue and explaining it in fairly concrete terms. I think that Mozilla is doing the right thing. We need open standards that are not patent encumbered. Keep fighting the good fight. Like many things its going to take the few that understand the issue to fight on the many that don't.
    Sean

    ReplyDelete
  104. Just to note that your final point about the possibility of MPEG-LA royalties for distribution (as opposed to encoding/decoding) of H.264 has been addressed - at least for the next 6 or so years:
    http://www.mpegla.com/main/Pages/Media.aspx
    This doesn't address the issues with the codec itself, of course, but given the weight of pressure for H.264 (in hardware) from the mobile, broadcast and IPTV industries I think it's going to be hard to resist. I do understand your rationale here, and it's a fine one, but I'm afraid this might be a battle you can't win.
    It's probably just worth adding that we are seeing IPTV set-top boxes using Webkit for UI very effectively now, and of course given they are sub-$100 low-power devices they are doing all the H.264 in hardware. At the moment they all have different proprietary ways of creating video players which means a lot of pain for us user interface developers. It would be great if we could develop to a standard HTML5 API. If this could then also work on the desktop _with the same content encoding_ then we're in heaven.
    This might just seem like a developer's dream world, but there is a serious point to this: In the UK at least, nearly 50% of households don't have Internet access at all. I think to enable those people to benefit from all the Web brings is going to require using the other two of the "3 screens" (TV and mobile), and if the same content can work seamlessly across all three, that goes a long way to enabling that.
    Put another way, there is a downside to insisting on a (currently) software-only codec - you are forcing people into using high-power, expensive general-purpose computers rather than cheap hardware to access it. Realistically, H.264 is the only codec with a chance of having that kind of hardware support for the forseeable future - and even getting that (instead of MPEG-2) took years.
    Best wishes
    Paul

    ReplyDelete
  105. Robert O'Callahan4 February 2010 23:02

    Paul, see http://www.schleef.org/blog/2009/11/11/theora-on-ti-c64x-dsp-and-omap3/

    ReplyDelete
  106. http://www.mpegla.com/main/Pages/Media.aspx
    H.264 is going to remain free for the next couple of years. In the interest of killing Flash I think you guys should go the same route as Google and implement .ogg and H.264.

    ReplyDelete
  107. Robert O'Callahan5 February 2010 02:17

    Free-as-in-beer, for Internet content providers, as long as they don't charge users, for a few years, yes. A smart move on the MPEG-LA's part, to increase the chances of the Web being completely locked into H.264, but not really "free".

    ReplyDelete
  108. a progress?
    http://www.atoker.com/blog/2010/02/04/html5-theora-video-codec-for-silverlight/
    silverlight doesn't have flash's market share and we want to get rid of those plugins... but it would at least remove the h.264 problem. one thing at a time...

    ReplyDelete
  109. As wonderful as this discussion may be about the merits of various codecs, their supportability, their cost, and their moral imperatives, I have yet to see a porn clip encoded in Ogg format. The popular choices seem to be WMV, DivX, Flash, h.264 (as MP4), and occasionaly there are variations of the ghastly Real format. Some of these formats may be more suitable for downloads than for streaming, but there is plenty of embedded content of this type out there. You should not forget this popular market segment when deciding what standards to support. Not everyone cares about YouTube videos of cats playing the piano.

    ReplyDelete
  110. This is why I like mozilla, they care about freedom!

    ReplyDelete
  111. "You should build a short text with a link to a more complete explanation of this issue that comes up every time the user tries to play a h264 video."
    We don't blam at user when they are browsing a non-standard HTML page right?

    ReplyDelete
  112. @flossy
    I'm talking about a screen that blames MPEG-LA, not the user!?! Where do you get that moronic idea that I would propose that the user gets a screen that says they are wrong. NO, I'm talking about a screen that says: "Because of those bad decisions... we can't offer support for that in a way it was supposed to be done. Feel free to tease the content owner about it. "
    Why do you think nobody knows what the problems are. Because nobody ever tells the user WHY they can't do something.
    The moment this retarded secrecy ideology is abandoned the better. This goes against the Open Source because you are actually helping bad people hiding their bugs. (See what happens in aliens)
    Again, this ain't the blame game, this is pointing where the problems are coming from for the user. This means that the user will be educated and eventually will react in a better way for an FOSS-ecosystem. Because they know what it means and what the alternative will mean.
    You do realize who are now seen as the stubborn or bad guys that don't integrate now? Gues you don't.

    ReplyDelete
  113. Brian Boyko, first the company for what you work have a big problem: They depend exclusively of a closed source software.
    That limit you a lot.
    You just said that Cinelerra does not give you what you expect. That will not be a problem if your company just pay a developer to implement the functionality what you need in cinelerra. And, why not? Extra functionality that the other software(Sony Vegas) actually does not include for your needs.
    Because the nature of Free Software, you can do that.
    So now, I ask you: Can you do the same with Sony Vegas or another propietary software? The answer is: No.
    So I ask you too: Are you free to work with propietary software? The answer: No, using propietary software your company are limited to the conditions of the developers of that software.
    So now I can say you: Don't wait OSS NLE to mature, just adapt it to your needs!
    That is because we prefer to use OGG Theora instead of patent issued software because you will be persecuted for using your natural righs of freedom. That sounds stupid, not? But is the reality.
    Bye

    ReplyDelete
  114. It's a great move from Mozilla to take a tough stand against this Trojan Horse HTML5/H.264 and support HTML5/Theora. If Google releases a patent-free Codec, you should think about to implement this together with Theora.
    But, please No H.264. As long as Firefox strictly support free codecs only, I support Mozilla.
    If Firefox ever can use H.264 for HTML5 I instantly would switch to another browser or fork which does not. Freedom of the Internet is more important.
    Keep the stance for freedom and free software. You're doing good.

    ReplyDelete
  115. stop whining and give the people freedom. freedom of CHOICE, so that we can choose to use non-free codecs.

    ReplyDelete
  116. Thank you for this post and the previous one about activex.
    For the record: in Israel (where I live) the government online payment service (which is facilitated by all government offices) does not work on Firefox (or other browsers I've tried)... it seems to only work with Internet Explorer...
    this means that I cannot make government payments from my linux (Ubuntu) based computers... I have to have access to a Windows + Internet Explorer computer to make these payments! so much for democracy, freedom and such...

    ReplyDelete
  117. As a video professional I think Mozilla is wrong here. Theora is not a format taken seriously at all by video professionals or amateurs using a mac or windows PC. In fact most people have never heard of it. Our software tools don't even give us the option to encode to Theora. It's an older format that doesn't produce as good of results compared to H.264. So we have no reason to pay it any mind.
    When I search popular professional video boards, I get one or two results for "Theora" that are several years old without any replies. Because nobody knows and nobody cares.
    Meanwhile H.264 is a standardized format we all use every day. More and more cameras are being released that capture to H.264. The AVCHD format is based on H.264. All of these new HDSLR's capture H.264. Every new smartphone comes with H.264 hardware decoding and you're starting to see PC's and netbooks with the same hardware decoding capabilities. Even Blu-ray plays back H.264. Add to that YouTube and Vimeo on the web.
    Video on the internet will be consumed on many different devices going forward. From desktops, laptops, smartphones, pmp's, web tablets and internet enabled TV's. So as a video professional we want to make sure that the videos we put out there can be consumed by as many people as possible. The format of choice is H.264. The video community has already decided this. We've already been working with this format for the last few years. This format is both compatible with current flash based video players and HTML5 browsers, sans firefox and opera (but nobody cares about opera).
    The idea that one open source software company is suddenly going to change all of that simply because of their principles, well that's just silly. Especially when the entire video industry is already on top of H.264 and top browsers like IE, Chrome, Safari, Android and iPhone browsers all support H.264.
    I'm sorry but you're going to lose here. I'm not saying I don't agree with your basis. I think it's notable. But it's not reality. Right now the only thing you are doing is holding back the HTML5 video tag from being used more widely. This will become even more apparent when IE9 comes out. Because I'll tell you right now, I'm not going through the time(equals money) and trouble to render out two versions of a video.

    ReplyDelete
  118. Very late to comment here, but I am _really_ worried that Mozilla grossly overestimates its ability to influence what codecs the big (bad) content providers use.
    Would it not be more expedient to use, for example GStreamer, but flash a warning when the format being played is encumbered? Possibly even link to a petition, or other forms of user action, to actually get sites to understand that users do actually care about these issues?

    ReplyDelete
  119. Regarding USA vs. the world: They already have their own web. For once I want to see the rest of the world unite and teach them a lesson.
    Many sites limit themselves to the US only (e.g. Hulu) so why should I care if the users of that site also need to pay for a codec license?

    ReplyDelete
  120. Where software patents do not apply, the patent holders lose nothing; where software patents apply, the patent holders lose nothing, but "because you live in the US," many people who use Firefox are excluded from a potentially important feature.
    Ok time to write to your congress man and start to have a minimal reasonable law on software patents, do we most of the rest of the world rule for you ?? you really have all my support. if it's anything we can do, just tell
    and you're wrong, do you think patent holders doesn`t waste pretty ammount of Euros in loobies at Europe, they REALY lose, they do...

    ReplyDelete
  121. Unfortunately for an average user, they just want their video to work. They won't care for software freedom if the video they want to see doesn't work in Firefox, they will switch to IE and continue their simple existence.
    I hope that they'll be able to sort out the licensing and standardize codecs for HTML5 that will be open source compliant but let’s face it, if worse come to worst, folks will jump the ship. Simplicity is the key for most users.
    I'll fight the good fight though :).

    ReplyDelete
  122. WalterBonono21 May 2010 20:19

    I think all the jibber jabber here is just zealot non-sense. I have used Firefox, IE, Safari and Chrome and frankly I find Firefox to be mediocre at best.
    I think it is pitiful people are unwilling to pay for anything software related. Developing is not free! Some poor bloke spent his time and money developing and just like any other product or service that is created, if it has value, I see no issue whatsoever in trying to make a business around it. Last time I checked Mozilla doesn't keep it's doors open with people that work for free or for the goodwill of mankind (let's be real please!)
    I really hate cheap asses who are unwilling to pay for anything that is digital. License the codecs and move on...

    ReplyDelete
  123. This is ridiculous.
    It is *not* ridiculous to want open source software. It is *not* ridiculous to promote Theora over H.264. It is *not* ridiculous to be concerned about the huge legal problems of H.264's patents.
    It is however *extremely* ridiculous to ignore the video processing abilities of the underlying OS. No matter how closed-source or obscure the codec is, if my OS has that codec available, I want the ability to make use of it.
    By all means, warn me about security risks before playing a video. Whitelist and blacklist to your heart's content. But never take this decision away from the user.
    I switched to Firefox because I really enjoy the Add-ons. In other words, I like the decisions I can make with it. If the developers insist on taking away my freedom of choice, no matter how noble their cause may be, I'm going to switch to Chrome or Safari.

    ReplyDelete
  124. Mozilla & Co. keep up the good work! You are the best!

    ReplyDelete
  125. Looks like all this might be changing now...

    ReplyDelete
  126. I think the mozilla reasoning here is rediculous, the codecs are ALREADY on my computer, so why won't it just use those??
    Gotta be a loophole here somewhere. Couldn't you stream the mp3 file packets, translate to ogg and then playback to the user, thus not breaking rules? Just an under-educated suggestion.
    Chrome is now my default, sorry :S

    ReplyDelete
  127. The arrogance in thinking you guys know best what we the users should or should not be using is appalling. The whole HTML5 video is starting to remind me of the bad old days where a site was best viewed with Internet Explorer or Netscape. Here are some reasons for me starting to take a hard look at what I use for video on the web (and I know I'm not the only one going through these kinds of decisions)
    1. You're forcing content creators to encode video material multiple times to accomodate your "freedom" ideals. In a limited server storage and development time world this is hard and forces people to decide what formats to use, usually ending up with the one that is supported by the most browsers/devices.
    2. You also don't consider the need to teach others what to use to encode the video makes it even harder. What do we use? How do we teach people to create the content?
    3. I've read all the arguments against using what's already installed on their systems. Isn't that the user's decision? From an end user perspective the browser should provide the technology to make my life easier not dictate what I should use to accomplish my goals.
    4. I have a ton of content already on mp4. By denying me the ability to play it natively in Firefox/Seamonkey you make it hard for me or my team to use open standards and promote the adoption of HTML5 videos since I'm already required to use Firefox as a supported browser where I work. My users just want the video to play, not have to worry about encoding and storage space for the new versions we'd have to create in order to accomodate Firefox or any open/free browser
    Freedom goes beyond using "free" technologies. It is about creating a user experience that doesn't limit or restrain what we do on the web. By limiting what you can do in "free browsers" like Firefox you're just limiting the market adoption of the technologies or the browsers themselves.
    It is a good theoretical exercise and I've created cross browser video displays. I am just not going to recommend it as a solution to my users as I don't want to increase my workload by an order of magnitude.

    ReplyDelete
  128. Agree here with Mozilla generally but wish they would fix the memory leak in FF4, had to switch back to 3.6 which is a shame.

    ReplyDelete
  129. Whether delivered by Mozilla or not, FF must be able to play mp3 audio and mp4/aac video from the html5 and tags.

    The next generation of rich media entertainment bundles are going to be delivered as "html5 apps", fully leveraging things like audio/video tags, local storage, ... These apps will be usable across the entire range of a user's connected technologies (desktops, laptops, media centers, smart phones, tablets, html capable TV's, ...).

    Flash "fallback" is no longer a viable alternative to html5 tag support, particularly with portable devices.

    Those who will create these bundles have pretty much spoken. Audio will be delivered as mp3, with possibly a lossless codec option. Video almost certainly be delivered as mp4 with aac audio. There is some discussion about adopting WebM as the video standard, but I don't believe it has legs.

    The open source stance of the Mozilla foundation on this issue won't change the direction being taken. The content creators will decide.

    In my mind, it's not a question of whether Mozilla delivers codec support directly (I understand the licensing issues), but that the support is readily available from somewhere.

    The lack of support for industry standard codecs can only hurt FF. Those of us who develop rich media players and library managers will be forced to abandon Gecko in favor of Webkit as our embedding vehicle.

    ReplyDelete
  130. I like Mozilla for the variety of its free add-ons and its high speed. But video loading can be sometimes a very long process. Seriously I was thinking it depended on the speed of my Internet connection, but it turns out that it's not only that. Thank you for the valuable information.

    ReplyDelete