Monday, March 2nd, 2009
Manipulating video in real time with Canvas
Category: Canvas
Paul Rouget has demonstrated "chroma-keying" (green screen effect) using JavaScript, the video tag, and canvas:
- The document establishes two canvas elements, with the IDs c1 and c2. Canvas c1 is used to display the current frame of the original video, while c2 is used to display the video after performing the chroma-keying effect; c2 is preloaded with the still image that will be used to replace the green background in the video.
- The JavaScript code is imported from a script named main.js; this script uses JavaScript 1.8 features, so that version is specified in line 22 when importing the script.
- When the document loads, the processor.doLoad() method in main.js gets run.
The work to do the image manipulation is:
JAVASCRIPT:-
-
computeFrame: function() {
-
this.ctx1.drawImage(this.video, 0, 0, this.width, this.height);
-
let frame = this.ctx1.getImageData(0, 0, this.width, this.height);
-
let l = frame.data.length / 4;
-
-
for (let i = 0; i <l; i++) {
-
let r = frame.data[i * 4 + 0];
-
let g = frame.data[i * 4 + 1];
-
let b = frame.data[i * 4 + 2];
-
if (g> 100 && r> 100 && b <43)
-
frame.data[i * 4 + 3] = 0;
-
}
-
this.ctx2.putImageData(frame, 0, 0);
-
return;
-
}
-
Fun stuff!
-
Related Content:
4.5 rating from 24 votes
Firefox 3.1 has some amazing performance. I’ve been doing a lot of tests between the Opera 10alpha, Safari 4, and Firefox 3.1, and am very happy with these next-generation browsers and their support of the tag. Seems like they all might be coming out within the next two months… get ready for some exciting new webapps :)
leptons –
great story. I believe you, I remember seeing a demo of something like that back in the day.
still – I’m fired up to try this at home tonight w/FF 3.1, and that this (video canvas element) will someday come to popular video sites instead of only via an add-on (flash).
@leptons: I’ve been looking for references to early IE work like that. Were there Direct3D bindings that could be used within the browser? Were these ActiveX controls? Have any docs or examples? Do they work anymore?
@Brad Neuberg:
This is from 2003 or 2004, and it’s IE only! hah
real-time video mixer with pixelation
http://www.atomicmandala.com
A Direct3D example – tested on IE 5.5 on WinME and IE6, texture-mapped 3D animation in the browser in 1999.
http://www.atomicmandala.com/direct3d/
Direct3D ended with IE6, it became a security risk or maybe silverlight killed off interest within MS for continuing to support it, and now it seems very difficult to find any information on the internet about Direct3D at all. I’d be very interested in seeing the docs for it if anyone knows how to access them.
That was awesome!!
The good stuff was Direct Animation. There was an exploit in the code and Microsoft killed it instead of fixing. I lost over a hundred scripts because of that.
.
Some good stuff:
http://www.mgifos.demon.co.uk/directanimation.htm
http://www.mgifos.demon.co.uk/vault/vault.zip
http://www.sworks.com/keng/da.html
thanks TNO, I meant to mention direct animation. wow, its been a long time since i’ve played with it. Yeah, there were some pretty cool demos of it before MS took it out in IE7.
IE does still support custom CSS filters, which is what I used for the pixelation effect on an embeded media player using the ‘windowlessVideo’ property to render the video without overlay – thereby allowing the video to render with the CSS filter effects. there are all sorts of other filter effects, chroma-key being one of them, although I don’t know if its that useful in IE, it uses only a single color. The matrix transform is interesting though, it can be used to rotate images/video (or any html element) in IE.
I’m glad canvas is supporting some of these things with video, it gives more possibilities for interesting cross-browser content.