Tuesday, June 9th, 2009

Finally, a useful blink tag! Detecting your user blinking

Category: Canvas, Showcase

Dave Burke just gave an awesome demo at GDD Beijing. Fire up Firefox 3.5beta (for now) and head over here and watch the human blink detection in action!

Here Dave tells us more about his awesome hack:

Inspired by a demo by Paul Rouget, I’ve created an image processing demo that detects eye blinks in real-time. It uses a combination of <video>, <audio>, <canvas> and worker threads. The <video> frames are “projected” on to a <canvas> to get access to the raw pixels. The Euclidean colour distance is calculated inter-frame and thresholded, resulting in a view that shows movement (shown on a second canvas). A worker thread then performs several algorithms including tracing, sorting, and positioning check of the blobs of pixels. If the worker thread determines an eye blink, it posts back to the main thread which uses the canvas to draw bounding boxes around the eyes. The algorithm is based on a paper I wrote almost 10 years ago describing a communication system I built for a patient with severe spinal injuries. I had originally implemented it in C++ using Microsoft’s “Video For Windows” (a precursor to DirectShow). I bet I would have laughed if someone said that I’d be able to write the same system in JavaScript less than 10 yrs later!

Thanks Dave!

Posted by Dion Almaer at 11:59 am
15 Comments

++++-
4.3 rating from 31 votes

15 Comments »

Comments feed TrackBack URI

Congratulations, you crashed my browser.

Also what is the point of having this functionality in a browser?

Comment by Darkimmortal — June 9, 2009

This is old news: http://labs.opera.com/news/2009/04/01/

:)

Comment by codedread — June 9, 2009

did you say…useful?

Comment by sixtyseconds — June 9, 2009

@sixtyseconds
@Darkimmortal
Technology like this is meant for accessibility. I imagine that if you cannot use your hands then stuff like this could be very useful. It says in the snippet that Dion posted: “…The algorithm is based on a paper I wrote almost 10 years ago describing a communication system I built for a patient with severe spinal injuries…”

Comment by turenando — June 9, 2009

This actually has the potential to be EXTREMELY useful… with the right mathematical gymnastics you could, I’d imagine, generate eye tracking diagrams and heat maps, which are very useful tools for the usability practitioners among us.

The fact that it’s done in HTML/JavaScript makes it all the more cool :)

By the way, I’ve been messing around with canvas an awful lot the past few weeks so keep anything related to it coming Ajaxian demi-gods! :)

Comment by fzammetti — June 9, 2009

@codedread
Errh…
The “Face Gestures” from Opera was their 1st of April joke this year dude …
Somehow I don’t think this is…

Comment by ThomasHansen — June 9, 2009

It crashes my Firefox 3.5b4 on WinXP in a blink of eyes, every time I access the page. How does my FF differ from the one of the author of the demo?

Comment by pmontrasio — June 9, 2009

Possibly the author is using a nightly build. It works fine for me in recent build.

Comment by doublec — June 9, 2009

Ok, but think of implementation here. It’s gotta use a video file on a live server. This means the user has to upload a video of themselves for the tech to be of real use (unless it’s possible to access local resources without the intervention of third-party scripts and severely lacking security)…

Comment by sixtyseconds — June 10, 2009

So…maybe very good for Titanium or AIR apps that have access to a webcam, but not so much actual websites/apps.

Comment by sixtyseconds — June 10, 2009

I’ve just updated FF to 3.5 Preview on Windows7 Beta and it works fine.

Comment by keemor — June 10, 2009

Motion detection with JavaScript… nice!

So in theory, lipreading software written in javascript? Possible?
Point webcam at monitor to track your hands and simulate a touchscreen?

tbh. with imagination, time and skills, this can lead to really sweet innovative web software.

Really hawt stuff Dave!

Comment by BenGerrissen — June 10, 2009

does anybody know if the html5 standard specify any interaction with web cameras? I mean, facial biometric identification can be performed on the browser with that.

Comment by nanodocumet — June 10, 2009

@Darkimmortal, heh…
1) If you don’t like your browser crashing…. try not using a beta browser?
2) This helps developers see the range of things we can do.

I have a great idea. We should just develop the same things over and over again, maybe it will stop your browser from crashing.

Congratulations for being an idiot.

Great article… amazing stuff.

Comment by drezdin — June 11, 2009

@drezdin – While this is an amazing bit of work, websites may not be the ideal target (wrt the current security model of browsers, and bandwidth considerations). That’s not to say we shouldn’t R+D, but this is possibly a bit before its time (in the browser at least)…

In terms of the article title though; this is not as useful as it is inventive.

Comment by sixtyseconds — June 12, 2009

Leave a comment

You must be logged in to post a comment.