HTML5: Manually "playing" a video inside a canvas, in IOS Safari

Question

So, here's what I'm trying to do: I want to load a video into a video element, but not have it played in the "normal" way. Using a timed interval, calculated according to the movie's framerate, I want on each iteration to

A. Manually advance the video one 'frame' (or as close as possible to that).
B. Draw that frame into a canvas.

Thereby having the video "play" inside the canvas.

Here's some code:

<video width="400" height="224" id="mainVideo" src="urltovideo.mp4" type="video/mp4"></video>
<canvas width="400" height="224" id="videoCanvas"></canvas>

<script type="text/javascript">

var videoDom = document.querySelector("#mainVideo");
var videoCanvas = document.querySelector("#videoCanvas");
var videoCtx = null;
var interval = null;

videoDom.addEventListener('canplay',function() {
   // The video's framerate is 24fps, so we should move one frame each 1000/24=41.66 ms
   interval = setInterval(function() { doVideoCanvas(); }, 41.66);
});

videoDom.addEventListener('loadeddata',function() {
  videoCtx = videoCanvas.getContext('2d');
});

function doVideoCanvas() {
  videoCtx.drawImage(videoDom,0,0);
  //AFAIK and seen, currentTime is in seconds
  videoDom.currentTime += 0.0416;
}

</script>

This works perfectly in Google Chrome, but it doesn't work in an Iphone's Safari;
The video frames does not get drawn at all to the canvas.

I made sure that:

• The video events I hooked into does get triggered (did 'alerts' and they were shown).
• I have control over the canvas (did a 'fillRect' and it filled).

[I also tried specifying dimensions in the drawImage - it didn't help]
Is drawImage with a video object not applicable at all in Iphone Safari...?

Even if I'll manage to find a way to capture the video frames, there are also some other issues in the Iphone's browser:

• Access to the currentTime property of the video is only granted once the video has started playing (in a standard way). I thought about maybe somehow "playing it hidden" and then capturing, but didn't manage to do that. Also thought of maybe somehow start playing the video and then immediately stop it;
  There doesn't seem to be any way to forcefully start the playing of a video in the IOS Safari. video.play(), or autoplay doesn't seem to do anything. Only if the user taps the "play circle" on the video then the video starts playing (taking all over the screen, as usually with videos on the IPhone's browser).

• Once the video plays - the currentTime property does get forwarded. On the video itself. When you pause the video and go back to the normal html page - you can see the frames on the video element changing. Though, in a slow phase (unlike in Google Chrome, where the rate seems to be smooth enough to make it look like it's playing) - in the iphone it looks to be a rate of something like 2-3 frames per second maybe. It stays the same even if I try changing the interval timing, I guess there's a minimum time limit that the browser on the IPhone can handle.

"Bonus question" :) - When the frames on the video element progresses from the event - the circle "play button" is visible on the video element (since it is not actually 'playing'). Is there anyway to hide it and make it invisible?

This has been tested on Iphone 3GS (with both 3G and Wifi) and Iphone 4, both running IOS 5, both having the same results as described.

Answer 1

Unfortunately I don't have an iOS device to test this, but I don't think you need to actually capture the video frames in the way that you're attempting using the currentTime property. The usual process looks something like this:

1. create a video element without controls (omit the controls attribute)
2. hide the element
3. when the video is playing draw it to the canvas on an interval

As you've discovered, iOS devices do not support autoplay on HTML5 video (or indeed audio) but you can create a separate control to initiate playback of the video using the play() method.

This approach should solve the issue you're having with the play button being visible since in this case you are actually playing the video.