Tuesday, 22 September 2009

mozLoadFrom And The Media Cache

In April I wrote about the Gecko media cache. Just recently I needed to make a significant extension to the media cache design to support a new experimental <video> element API: mozLoadFrom.

v1.mozLoadFrom(v2) behaves much like the load method of HTMLMediaElement. It stops playing the currently loaded resource and loads a new resource. However, it does not run the HTML5 media selection algorithm; instead it just takes v2's currentSrc and loads that. What makes mozLoadFrom useful is that it tries to avoid re-downloading the resource data. It grabs all the cached and buffered data already associated with v2 and makes it available to v1 as well. In fact these elements carry on sharing data, so that any future data downloaded by v1 or v2 is available to the other. Typically this means that only one of the elements will actually be downloading data and the other one won't need to download anything at all. (Although if the entire resource doesn't fit in the media cache, you can end up in situations where the elements are playing different points in the stream and both reading from different offsets.)

mozLoadFrom is useful if you need to manipulate multiple views of the same video/audio stream. For example you might be playing a video and at the same time you want a script to seek forwards or backwards and gather periodic snapshots into a series of <canvas>es. Or you might have a set of videos playing in the page at a smallish size and want to display a selected one in the front covering most of the window, without having to mess with the original thumbnail video. The main reason I implemented mozLoadFrom is to make it easier to display full-screen video in Firefox or extensions.

The tricky part of the implementation is that the media cache was originally designed so that a block in the cache could only belong to one decoder at a time. With mozLoadFrom a block can be owned by multiple decoders at a time. In the original design, a block is always in exactly one of four states: free, readahead (it's ahead of the current playback point), played (it's behind the current playback point), and metadata. Now a block can be a readahead block for one decoder and a played block for another decoder.

I was quite worried it would be hard to extend the design this way, but in fact I was able to do it in about a day of intense hacking. This probably reflects well on the original design of the media cache. In particular, basing eviction decisions on the "predicted time of next use" was easy to extend; when a block has multiple owners, its predicted time of next use is the earliest time that any of its owners predicts for it.

Update Ian Hickson points out that new API is not needed here since HTML5 allows concurrent loads of the same absolute URI to share data regardless of HTTP caching instructions. So instead of mozLoadFrom we could just use v1.src = v2.currentSrc to get the same effect. I should do that...



2 comments:

  1. Per the HTML5 spec, just creating a new and doing newvideo.src = firstvideo.currentSrc; should have the same effect. (Well, may have. Implementations aren't _required_ to use the same streams, but they're allowed to.)

    ReplyDelete
  2. Robert O'Callahan22 September 2009 09:50

    Hmm, indeed. I didn't know that section 2.6 lets us ignore HTTP caching semantics for cases like this. Great!

    ReplyDelete