Dave Burke just gave an awesome demo at GDD Beijing. Fire up Firefox 3.5beta (for now) and head over here and watch the human blink detection in action!
Here Dave tells us more about his awesome hack:
Inspired by a demo by Paul Rouget, I’ve created an image processing demo that detects eye blinks in real-time. It uses a combination of <video>, <audio>, <canvas> and worker threads. The <video> frames are “projected” on to a <canvas> to get access to the raw pixels. The Euclidean colour distance is calculated inter-frame and thresholded, resulting in a view that shows movement (shown on a second canvas). A worker thread then performs several algorithms including tracing, sorting, and positioning check of the blobs of pixels. If the worker thread determines an eye blink, it posts back to the main thread which uses the canvas to draw bounding boxes around the eyes. The algorithm is based on a paper I wrote almost 10 years ago describing a communication system I built for a patient with severe spinal injuries. I had originally implemented it in C++ using Microsoft’s “Video For Windows” (a precursor to DirectShow). I bet I would have laughed if someone said that I’d be able to write the same system in JavaScript less than 10 yrs later!
You will notice that the UI has an iTunes-like feel with the bottom timeline, and it looks great. I am starting to wonder if you have to get a UI review from Sofa before you deploy a Cappuccino application? What are you hiding 280North! ;)
"darkimmortal" has created another really nice visualization of music that uses Canvas and SoundManager2 to do its work.
Do yourself a favour, and hit play to see the funk
The example uses two canvases, one to do the logic wave work, and then it is copied into the main canvas that you see ctxR.drawImage( ctxL.canvas, 0, 0 );.
The Crap I missed it! crew took on the task of dealing with importing your iTunes XML file, and wanting to give you responsive feedback on the items as they come in. The usual tactic would be to suck in the entire file, and then process it.
Michael Baldwin did more, and here he tells us more:
We wanted to allow users to upload their iTunes library files, so that we could extract artist names (to let users sign up for new album and concert notifications). The problem was that these .xml library files can easily run up to 20MB in size.
Which means 1) long, boring downloads without feedback that it's really working, 2) huge space requirements on our servers to support lots of concurrent uploads, and 3) big memory requirements to process the XML files.
What we did instead was to write a bare-bones custom web server just for this task in PHP (yes, PHP) which analyzes the file as it streams in (storing nothing on disk, and using negligible memory), gradually puts artist statistics into shared memory, and then responds to new AJAX requests every ten seconds which retrieve and remove the artist statistics from shared memory.
The end result for users: as users upload their library file over the course of several minutes, they get to watch their web page fill up with their list of artists at the same time, sorted and even animated. If the connection breaks, they can even choose to continue with just the artists that made it.
The end result for us: we can deal with gigabytes of uploads while using trivial computing resources -- just a couple KB of incoming buffer space and a couple KB of outgoing buffer space.
You can check it out yourself, where it will look a bit like this (but constantly updating!)
David Semeria has been working on LM, another Ajax framework, for a number of years and has just announced it.
The first demo app is a highly customizable Twitter client, Twiggler, that runs in the browser.
Unfortunately nothing is public on the framework so we can't see how it actually works. It looks kinda like Squeak or X from an interface perspective, and you can see an interesting designer view.
Ben loves to talk about old games such as Dark Castle. I personally love remembering old Sinclair ZX Spectrum games but also fondly remember Leisure Suit Larry.
Wouldn't it be nice if you could play classic old games such as these easily online? Martin Kool thought so, and wanted to create an open source adventure game system, and just released Sarien.net "the portal for reliving the classic Sierra On-Line adventure games. With its focus on instant fun and a unique multiplayer experience, Sarien.net hopes to win new gamers' hearts and promote the adventure game genre."
Features
Completely browser based, runs in IE, FF, Opera, Safari, Chrome
Works on the iphone and Wii (both needs fixing at the moment I believe)
Games are completely playable using mouse or keyboard input. A lot of effort has been put into the user interface to make it work for the era that we’re in
Multiplayer! This was not in the original games, but what you can do now is; when you are playing your own game at your own pace, and are in game X room Y, you’ll see other players in that same game X room Y doing their things
You can talk to eachother and replay scenes together
All game areas are accessible directly through the address bar, and bookmarkable. Giving players the instant kick of replaying a certain scene
While playing, you encounter other npc’s. These add up to your own list of avatars that you can choose to be, not influincing the gameplay but it adds up to the fun (especially in multiplayer)
There’s even an extra Studio tool to let you view all game resources, similar to existing tools such as AGI Studio
SitePen has created Stocker "which demonstrates some of the more advanced capabilities of Dojo, including the newly released DataChart, the DataGrid, Data Store, Comet, Persevere, and BorderContainer. SitePen is also offering a one-day workshop where you will learn how to create Stocker yourself, but I’m here to give you a sneak peak of what Stocker is and how it works.
Stocker uses these technologies to emulate a stock monitoring application. We’re using made up data, but that’s actually more interesting. The Persevere server generates new stock items at certain intervals, and then pushes them to the browser with Comet. Then the Data Store updates its items and triggers an onSet notification. The DataGrid and DataChart are both connected to the same store, and are listening to that event. They then update their displays and show the stock items and their latest data."
Want to see what it would be like to develop using Persevere, DataGrid, DataChart, and friends!
Just last week a few of us were discussing the riches that await discovery in the various computer science research paper archives and today Peter Bergström wrote in to tell us that he's finished work on his PaperCube research paper search engine (which we mentioned when it was incomplete some time ago).
Bsaed on SproutCore, PaperCube lets you easily explore papers, including linking to references and citations--its very cool, and Peter's prepared a video to walk through the features:
Jacob Seidelin is at it again, with another music visualization using canvas and more:
A couple of weeks ago I played around with music visualization using JavaScript/canvas and SoundManager2. Well, I couldn't leave it at that and as I mentioned in the comments, I had an eye on the MilkDrop plugin for Winamp. The result so far is a little Winamp lookalike called JuicyAmp with its own music visualizer JuicyDrop that feeds on Milkdrop preset files.
If you just want to see the pretty colors -> CLICKY. (But please use Chrome or Firefox 3+)
MilkDrop is nice because, although there's a built-in editor in the plugin, the presets are in plain text. They are basically just lists of variables and equations that, with a bit of mangling, can be evaluated as JavaScript. There are also extensive guides that explain how to author presets and how variables are passed around between the different equations. And, even better, the source code for the plugin was released a couple of years ago.
MilkDrop presets consist of a number of different elements (waveforms, shapes, per-pixel effects, etc). Some of them I haven't touched at all, but JuicyDrop supports enough at the moment that a good handful of presets run just fine. That said, there are a whole bunch of problems to work out and the presets included were selected because they looked alright and didn't make it blow up.
I strongly recommend using Chrome for this. Firefox 3 can play too but is probably somewhat slower. There's something screwy going on with Safari, it's like it's refusing to update the display (try holding down a key on your keyboard), I'm not sure what exactly is causing it. Opera is a mixed experience for me and it seems to have a problem with playing the music that I haven't found the reason for yet.
Project Dragonfly is the latest experiment from the Autodesk Labs group. It enables to visually floor plan in both 2d and 3d space:
A few of the folks who brought you Project Draw and Autodesk Seek, teamed with some fairly new members of the Autodesk family, have cooked up something new for the home project "do-it-yourselfer" - Project Dragonfly. I grew up in Louisiana and we used to call them Mosquito Hawks, so I was happy to hook this free web-based floor planning application into the Autodesk Labs site. Project Dragonfly’s technology preview offers all-types of users, even those without prior design skills, the ability to visualize a future home improvement project or dream about a long awaited one.
The team told us a little of the implementation:
In terms of implementatoin we use the Flex platform for the client. We are using a very cool 2d library for flex called Degrafa and Away3d for the 3d implementation. The most interesting aspects of this application IMHO are the rules system which provides ability for collision detection (this is not just a bounding box collision but a true geometry collision), orientation (objects reorient as they move closer to walls) and ability to do wall attachments.
On the 3d front of course the quality of user experience and performance were equal concerns. So we heavily customized the away3d framework, especially the renderers and offloaded a ton of work to the server (we use the amazon computing cloud-ec2 for hosting, s3/sdb for storage and cloudfront for CDN) so we can handle the processor intensive tasks away from the client.
Sigurd Magnusson recently told us about ModelAdmin, a just-released feature in the open source SilverStripe CMS that uses jQuery to build rich user interfaces to let every day people manage data in a web-app.
This free code will let strong PHP+JS developers build really great web interfaces to be used by non-technical people.
You write object-oriented MVC code in PHP, and a jQuery-based application, along with the database schema, automatically builds around you. (This is similar to ruby on rails, with the exception that the database schema is derived from code, rather than the code inspecting the database schema.)
One of the developers of the work, Ingo Schommer, has a 25 minute video tutorial on this
The SilverStripe CMS was featured on Ajaxian 24 months ago for starting a new era in terms of CMS usability, using scriptaculous and prototype. The system is being migrated to jQuery, to give more power to developers. ModelAdmin one of the first items across the line.
I saw two posts at the same time on the topic of multiple-file uploading from a Web app which is a topic that wehavecoveredmanytimesbefore.
This time we have our good friend Scott Schiller posting on the Flickr uploader. He details the user experience from file selection, to progress, to completion, and then delves into the technical details including the Flash 10 changes:
Due to a change in the security model beginning with Flash 10, file selection must now begin via the user clicking directly on the Flash movie. With previous versions, you could call [Flash movie].selectFiles() from JavaScript and the selection dialog would be shown without requiring user action.
To keep the user experience consistent on Flickr where an HTML link could trigger the selection dialog, we made the Flash movie transparent and overlaid it on top of the HTML link. Thus, the Flash movie captures the click and the selection dialog works as expected. One might call this a legitimate, local form of clickjacking.
If repositioning the Flash movie is undesirable in your use case, another option is to render the link, text or button UI within the Flash movie itself and show the movie as a normal inline element.
Ryan Doherty has one of those really nice articles that walk you through how to do something that should be easy in CSS but isn't, until you know how: Cross Browser Inline Block.
SitePen, via Adobe sponsorship, has created an open source queue manager for Netflix called Queued.
The entire Dojo toolkit contains much more functionality than any single application needs, so the task of building applications often comes down to figuring out what style of coding you’ll use rather than learning how to use different components and figuring out how to glue them together. For Queued, we ended up with:
a single HTML file for the main window
dAIR for Dojo/AIR integration (window handling, etc)
dijit for layout (BorderContainer and friends)—but only for layout; we wanted to show that you can build compelling experiences without having to include (or learn!) the entire toolkit
unobtrusive behavior implementation using dojo.behavior, which made it very easy for our design & CSS guys to be productive without conflicting with the JS guys (and vice versa)
dojox.dtl for most widget templating
drag and drop for queue re-ordering
various animations for polish
Dojo’s build system (we distill everything into a single dojo.js and qd.js for the production app)
Since Queued is as much a demonstration of Dojo as it is of AIR, we took special care to keep the code hackery to a minimum—in addition to simply working correctly, we needed the code to be 1) easy to follow and 2) instructive for developers interested in learning how a good Dojo app can be put together. Everything’s nice and organized.
