html5-canvas Media types and the canvas Basic loading and playing a video on the canvas.
Example
The canvas can be used to display video from a variety of sources. This example shows how to load a video as a file resource, display it and add a simple click on screen play/pause toggle.
This stackoverflow self answered question How do I display a video using HTML5 canvas tag shows the following example code in action.
Just an image
A video is just an image as far as the canvas is concerned. You can draw it like any image. The difference being the video can play and has sound.
Get canvas and basic setup
// It is assumed you know how to add a canvas and correctly size it.
var canvas = document.getElementById("myCanvas"); // get the canvas from the page
var ctx = canvas.getContext("2d");
var videoContainer; // object to hold video and associated info
Creating and loading the video
var video = document.createElement("video"); // create a video element
video.src = "urlOffVideo.webm";
// the video will now begin to load.
// As some additional info is needed we will place the video in a
// containing object for convenience
video.autoPlay = false; // ensure that the video does not auto play
video.loop = true; // set the video to loop.
videoContainer = { // we will add properties as needed
video : video,
ready : false,
};
Unlike images elements videos don't have to be fully loaded to be displayed on the canvas. Videos also provide a host of extra events that can be used to monitor status of the video.
In this case we wish to know when the video is ready to play. oncanplay means that enough of the video has loaded to play some of it, but there may not be enough to play to the end.
video.oncanplay = readyToPlayVideo; // set the event to the play function that
// can be found below
Alternatively you can use oncanplaythrough which will fire when enough of the video has loaded so that it can be played to the end.
video.oncanplaythrough = readyToPlayVideo; // set the event to the play function that
// can be found below
Only use one of the canPlay events not both.
The can play event (equivalent to image onload)
function readyToPlayVideo(event){ // this is a referance to the video
// the video may not match the canvas size so find a scale to fit
videoContainer.scale = Math.min(
canvas.width / this.videoWidth,
canvas.height / this.videoHeight);
videoContainer.ready = true;
// the video can be played so hand it off to the display function
requestAnimationFrame(undateCanvas);
}
Displaying
The video will not play itself on the canvas. You need to draw it for every new frame. As it is difficult to know the exact frame rate and when they occur the best approch is to display the video as if running at 60fps. If the frame rate is lower then w just render the same frame twice. If the frame rate is higher then there is nothing that can be don to see the extra frames so we just ignore them.
The video element is just a image element and can be draw like any image, you can scale, rotate, pan the video, mirror it, fade it, clip it and display only parts, draw it twice the second time with a global composite mode to add FX like lighten, screen, etc..
function updateCanvas(){
ctx.clearRect(0,0,canvas.width,canvas.height); // Though not always needed
// you may get bad pixels from
// previous videos so clear to be
// safe
// only draw if loaded and ready
if(videoContainer !== undefined && videoContainer.ready){
// find the top left of the video on the canvas
var scale = videoContainer.scale;
var vidH = videoContainer.video.videoHeight;
var vidW = videoContainer.video.videoWidth;
var top = canvas.height / 2 - (vidH /2 ) * scale;
var left = canvas.width / 2 - (vidW /2 ) * scale;
// now just draw the video the correct size
ctx.drawImage(videoContainer.video, left, top, vidW * scale, vidH * scale);
if(videoContainer.video.paused){ // if not playing show the paused screen
drawPayIcon();
}
}
// all done for display
// request the next frame in 1/60th of a second
requestAnimationFrame(updateCanvas);
}
Basic play pause control
Now we have the video loaded and displayed all we need is the play control. We will make it as a click toggle play on the screen. When the video is playing and the user clicks the video is paused. When paused the click resumes play. We will add a function to darken the video and draw an play icon (triangle)
function drawPayIcon(){
ctx.fillStyle = "black"; // darken display
ctx.globalAlpha = 0.5;
ctx.fillRect(0,0,canvas.width,canvas.height);
ctx.fillStyle = "#DDD"; // colour of play icon
ctx.globalAlpha = 0.75; // partly transparent
ctx.beginPath(); // create the path for the icon
var size = (canvas.height / 2) * 0.5; // the size of the icon
ctx.moveTo(canvas.width/2 + size/2, canvas.height / 2); // start at the pointy end
ctx.lineTo(canvas.width/2 - size/2, canvas.height / 2 + size);
ctx.lineTo(canvas.width/2 - size/2, canvas.height / 2 - size);
ctx.closePath();
ctx.fill();
ctx.globalAlpha = 1; // restore alpha
}
Now the play pause event
function playPauseClick(){
if(videoContainer !== undefined && videoContainer.ready){
if(videoContainer.video.paused){
videoContainer.video.play();
}else{
videoContainer.video.pause();
}
}
}
// register the event
canvas.addEventListener("click",playPauseClick);
Summary
Playing a video is very easy using the canvas, adding effect in real time is also easy. There are however some limitations on formats, how you can play and seek. MDN HTMLMediaElement is the place to get the full referance to the video object.
Once the image has been drawn on the canvas you can use ctx.getImageData to access the pixels it contains. Or you can use canvas.toDataURL to snap a still and download it. (Only if the video is from a trusted source and does not taint the canvas).
Note if the video has sound then playing it will also play the sound.
Happy videoing.
