Billy Lamberta got tired having to get so low-level as you do with the raw Canvas API. So, he create a small abstraction layer:
Doodle.js attempts to create a fun and easy way to interact with the Canvas that is lightweight, flexible, and functional. While it contains a few shape primitives it is not meant to be a full-fledged drawing api, rather a framework that allows you to build sprites and interact with them in an expressive way.
He gives plenty examples, but here is one that animates the spiral that you see:
}, 0, '48fps'); //loop indefinitely at 48 frames/sec
Best part of the framework? How he choose the "oo" convention for the main object:
The “oo” variable was picked because it’s easy to type, easy to remember (d-oo-dle), and easy for our eyes to parse when we look over the code (it only takes a very cursory knowledge of male psychology to understand that).
I was actually surprised that I hadn't already posted on this... or CAKE which I will mention next...
Dave Burke just gave an awesome demo at GDD Beijing. Fire up Firefox 3.5beta (for now) and head over here and watch the human blink detection in action!
Here Dave tells us more about his awesome hack:
Inspired by a demo by Paul Rouget, I've created an image processing demo that detects eye blinks in real-time. It uses a combination of <video>, <audio>, <canvas> and worker threads. The <video> frames are "projected" on to a <canvas> to get access to the raw pixels. The Euclidean colour distance is calculated inter-frame and thresholded, resulting in a view that shows movement (shown on a second canvas). A worker thread then performs several algorithms including tracing, sorting, and positioning check of the blobs of pixels. If the worker thread determines an eye blink, it posts back to the main thread which uses the canvas to draw bounding boxes around the eyes. The algorithm is based on a paper I wrote almost 10 years ago describing a communication system I built for a patient with severe spinal injuries. I had originally implemented it in C++ using Microsoft's "Video For Windows" (a precursor to DirectShow). I bet I would have laughed if someone said that I'd be able to write the same system in JavaScript less than 10 yrs later!
"darkimmortal" has created another really nice visualization of music that uses Canvas and SoundManager2 to do its work.
Do yourself a favour, and hit play to see the funk
The example uses two canvases, one to do the logic wave work, and then it is copied into the main canvas that you see ctxR.drawImage( ctxL.canvas, 0, 0 );.
Canvas is a very simple API, much like what we’ve proposed to Khronos for 3D support. It’s well-scoped, well understood and integrates very well with other web technologies. And it’s been getting a huge amount of traction on the web. People are writing all kinds of really neat technology on top of it, including useful re-usable libraries for visualization. Have a look through Google’s own promotional site for Chrome - a huge number of them use canvas. It has traction. And we’ve gone through a couple of iterations - we’ve added support for text and a couple of other odds and ends once we understood what people were trying to do with it.
Now compare this to SVG and SMIL. Each of those specs are multi-hundred page documents with very large APIs and descriptions of how to translate their retained-mode graphics into something that’s usable on the web. (SVG 1.1 is a 719 page PDF. SVG 1.2 Tiny is 449 pages. The spec for SMIL is a 2.7MB HTML file.) We’ve seen some implementation of SVG and SMIL in browsers, but it’s been slow in coming and hasn’t seen full interoperability testing nor any real pick up on the web. The model for these specs was wrong, and I think it shows.
Chris doesn't directly say that Google's approach is "wrong", but he wonders if the Google proposal of a bigger and more ambitious API would represent too great a compatibility burden for browser vendors and developers.
In the comments of his post, Henry Bridge of the Google O3D team replied; here's a lightly edited excerpt:
We agree that to keep a standards process focused, APIs should be as minimal as possible while remaining useful, and so we would likely keep things like that out of any first attempt at a standard and, as you say, let it evolve over time. But the usefulness question brings up an important, and we think, unresolved point. We’d love to build the animation and skinning system in JS, but we just couldn’t get a JS-based animation system fast enough — even on our retained-mode API. Javascript is getting faster all the time and we love that, but until someone builds some apps it’ll be hard to know what’s fast enough.
Standardizing [an Open GL-like] immediate mode API for JS makes total sense. It’s a well defined problem, lots of people know GL, and we think it will be useful. But some of the demos we wrote _already_ don’t run well without a modern JS implementation, and moving to [Open GL] won’t help that (but we’d love to be proven wrong). That’s why we think it makes sense to explore both an immediate and a retained mode 3D, and make sure they work well together.
Paul Rouget and Tristan Nitot are having a lot of obvious fun with Canvas and <video> these days. The latest goodness is a demo by Paul that shows real-time dynamic content injection.
Notice the Firefox logo in between the two phones with bright screens? That is injected into the world thanks to Canvas.
How did Paul do this? He told us:
Obviously, I use the <video/> tag.
But what you see is not the video element (display: none;), but a <canvas/> tag.
All the patterns you see on the top right are regular <img/>,
<video/> and <canvas/> elements. The play/pause button is
a <svg/> element
(position: absolute;) on the top of the main <canvas/> element.
A canvas element provides a method named drawImage which let you
inject the content of a DOM element in the canvas (like a screenshot). It works
with three kinds of elements: <img/>, <canvas/> and
<video/>.
When you click on the <svg/> button, the Javascript code launches the
main video. Then, the main javascript loop is executed each 10
milliseconds.
Here are key things that occur during the main loop:
first, the content of the video is injected in the main canvas. That's why
the canvas element looks like a video element;
second, the position of the 2 brighter areas of the canvas are computed
(you have access to all pixels values);
third, the required transformation is computed (rotation, scale,
translation);
fourth, the content of the selected pattern is injected in the main canvas
following the transformation.
Nicely done guys. If we ever get a sort algorithm question again not only will we be able to give the big O notation, but imagine pulling out some Canvas code to visualize ;)
Last week, we posted a story on the Metatunnel demo, a concisely-written Linux and Win32 demo that Paulo Falcão ported to JS. We looked at the pitiful performance of the JS version as evidence that, high-speed JS engines notwithstanding, the web's got a ways to go on the fx front.
We didn't dig deep enough.
Someone posted a follow-up to our story saying that the JS code was poorly optimized and that the JS version could be made to run faster. That got me wondering how efficient the desktop version was, and how apples-to-apples the versions were in terms of total calculations performed per frame. My thinking was that if the JS version could be made faster but via optimization contortions that the desktop version didn't require, that's missing the point.
It turns out the desktop source is trivial: it's an assembler bootstrap for passing OpenGL shader instructions to the system's OpenGL driver:
Shoot, passing instructions to the video card? That's not a fair fight. It turns out Firefox 3.5 has a prototype Canvas 3D plug-in, and it exposes an OpenGL API. It was quick work for one of Mozilla's graphics gurus and the plug-in's primary developer, Vlad Vukicevic, to port the Metatunnel demo to Canvas 3D. The result is quite impressive:
We should also mention that Paulo, Jacob Seidelin, and Hans Schmucker collaborated on optimizations in the JS software-rendering version that resulted in much higher FPS on "next-gen JS" browsers (Safari 4, Chrome, and Firefox 3.5). Note, on this demo you can use the Q and W keys to adjust quality.
Dion and I talk frequently about how the web platform's capabilities are approaching the desktop, which features like canvas and faster JavaScript run-times. It's great to get a piece of humble pie like this one.
Fabien Ménager has created an interesting new library called canvas-text that simply "adds the three canvas text functions (strokeText, fillText and measureText) to the canvas implementation of the browsers which don't already have these functions (Firefox 2/3.0, Internet Explorer 6+, Opera 9+, Safari 3, Chrome 1.0). It doesn't change the already implemented functions in Firefox 3.1+, Safari 4 and Chrome 2."
Having the methods work is one thing, but what about the performance?
Well, there are some interesting examples for you to test:
I talked to Fabien about performance and he shed some more light:
The only bad performance issues are the font face download, JSON eval,
and data parsing. I try to use a cache when possible, but it's not
always possible.
A workaround would be to have two font face files : a full one, with
lots of characters, and a reduced one, with only the common ones. The
browser cache does the rest for the font face file download
(downloaded via Ajax or a script tag).
For IE, I think I'll have to make a VML renderer for IE that wouldn't send the
instructions to ExCanvas but would work directly on the VML.
Obviously, this is something I really want to play with and see how far we can get with this type of thing and Bespin. It is very much in an alpha state right now, but looking good.
Gauge.js 1.0 allows you to add programmable gauges (with shading and reflection) to your webpages.
It uses unobtrusive javascript to keep your code clean.
It works in all the major browsers - Mozilla Firefox 1.5+, Opera 9+, IE6+ and Safari.
On older browsers, it'll degrade and your visitors won't notice a thing.
I have been very impressed with Andrew Sutherland of the Mozilla Messaging team (which is one reason that I have faith that I will dump Gmail for something he and the Messaging team come up with one day ;) and the visualizations that he is playing with are quite cool indeed. Seeing your email in different ways (not just in a table) can open the eyes.
Andrew is using Protovis "a visualization toolkit for JavaScript using the canvas element. It takes a graphical approach to data visualization, composing custom views of data with simple graphical primitives like bars and dots. These primitives are called marks, and each mark encodes data visually through dynamic properties such as color and position."
We’ve started to see more and more libraries being built to support use cases with Canvas in a 2D context but we really want to take things to the next level and start to allow people to use 3D capabilities as well. Accelerated 3D graphics with the super-fast next-generation JavaScript engines from nearly every web browser vendor means that we’re going to be able to start to see more and more advanced applications written using open web technologies. 3D is a huge part of that story and we’re happy to bring our proposal to the table.
The proposed spec (found in one of vlad’s post on 3D Canvas) is a pretty light wrapper on top of OpenGL ES 2.0, with some changes to support some JavaScript pleasantries. OpenGL ES is a decent starting point, which is why we picked it. OpenGL is supported as part of every major operating system and in it’s being picked up as a standard on mobile devices as well. Compared to the full OpenGL spec, the ES variant is a smaller subset that reflects the reality of what’s being used on the ground and most hardware and software vendors have actually been re-tooling to support OpenGL ES with support for older versions of full OpenGL emulated on top of OpenGL ES. Mixed with the fact that there’s a decent amount of knowledge out there in the industry of how to use OpenGL, we think that this smooths the integration between the current set of OpenGL users and larger web developer community.
And, there is the link back to Vladimir Vuki?evi?, the engineer who did the initial work (and who has been a huge help for Ben and I at Mozilla wrt Canvas and much more).
This is great stuff, and is it is important to clarify that this isn't about 3D virtual worlds, like some people think. This isn't VRML. The iPhone and the Mac are doing really nice 3D effects all the time these days. Close a window in OS X / Vista. Launch and quit Time Machine.
I am also glad to see OpenGL ES instead of yet another attempt at doing the 3D. There are a lot of tools and mindshare around this, and people will build great libraries on top of it I am sure for particular niches (apply some effects etc) and maybe we can get some uplift to CSS itself :) Oh, and i am really glad to see Google involved too!
I also saw Ryan Stewart weighing in and liked some of what he had to say, but was surprised by:
So it’s unfortunate to see that even the browser vendors have given up on moving the open web forward through standards. Whether it’s the WHATWG versus the W3C or the trials and tribulations of actually implementing HTML5, things are very broken and everyone is moving on regardless. I don’t blame any of them, but it doesn’t seem like it’s good for web developers.
Standards groups aren't the place to innovate. Browsers should be creating compelling features, other browsers should copy the good ones, and then we can standardize. XMLHttpRequest wasn't grown in the W3C. The cool CSS ideas that have been implemented in multiple browsers recently weren't either. Browsers need to push more, not less. And, then we need the standards groups to rally and try to turn the -vendor-bar-baz into bar-baz. Giving Web developers more APIs is good for Web developers!
Jacob Seidelin is at it again, with another music visualization using canvas and more:
A couple of weeks ago I played around with music visualization using JavaScript/canvas and SoundManager2. Well, I couldn't leave it at that and as I mentioned in the comments, I had an eye on the MilkDrop plugin for Winamp. The result so far is a little Winamp lookalike called JuicyAmp with its own music visualizer JuicyDrop that feeds on Milkdrop preset files.
If you just want to see the pretty colors -> CLICKY. (But please use Chrome or Firefox 3+)
MilkDrop is nice because, although there's a built-in editor in the plugin, the presets are in plain text. They are basically just lists of variables and equations that, with a bit of mangling, can be evaluated as JavaScript. There are also extensive guides that explain how to author presets and how variables are passed around between the different equations. And, even better, the source code for the plugin was released a couple of years ago.
MilkDrop presets consist of a number of different elements (waveforms, shapes, per-pixel effects, etc). Some of them I haven't touched at all, but JuicyDrop supports enough at the moment that a good handful of presets run just fine. That said, there are a whole bunch of problems to work out and the presets included were selected because they looked alright and didn't make it blow up.
I strongly recommend using Chrome for this. Firefox 3 can play too but is probably somewhat slower. There's something screwy going on with Safari, it's like it's refusing to update the display (try holding down a key on your keyboard), I'm not sure what exactly is causing it. Opera is a mixed experience for me and it seems to have a problem with playing the music that I haven't found the reason for yet.
Erik Arvidsson, one of our Knights, has shared a new Explorer Canvas release that has enabled excanvas folks to keep on trucking wrt IE 8 users. The way IE 8 does VML has been tweaked, so the library had to be changed accordingly.
Implement transform and setTransform. This passes all the tests in the HTML5 canvas tests for 2d.transformation.*
Remove fallback content that caused some issues when resizing the canvas element.
Fix issue where strokeRect, fillRect and clearRect incorrectly cleared the current path.
Added 2 new tests and modified an existing test to ensure that the new code works as expected and tested this in all modern browsers
Fix for IE8. This involved passing one more argument to namespaces.add as well as ensuring all CSS lengths have units (px). Passes all the test cases in all modes in IE8.
Fixes for linear gradients.
Added two test cases for linear gradients.
Changed the calculation method of line width. An averaged line width is calculated from the determinant of matrix, which is valid even when transform() method is implemented someday.
Improved the rendering of lines. Lines with the width less than 1px look better now.
Fixed the bug that stroke() ignored lineCap, lineJoin and miterLimit when fillStyle attribute was set.
Removed the settings of strokeweight, strokecolor and fillcolor. They are unnecessary since they are overridden by the weight and color attributes in <v:stroke> and <v:fill>. </v:fill></v:stroke>
This fixes issues where translate, rotate and scale is used during a path is being constructed. Previously we did the coord translations just before we draw the path. That is incorrect and now we do the translations when we add each individual piece to the path
Added very limited support for scaling of the stroke width. Currently we do the scaling by calculating the position in the final coordinate space and we therefore cannot do non uniform scaling of the stroke. For now we just do the max x/y scale factor.
Fix stroke. It should not close the path
Fix memory leaks
Fix issue where the path was not closed when strokeRect/fillRect was called.
Use the document.createElement('canvas') hack that was exposed by Sjoerd Visscher last week. This allows us to remove fixElement_ completely.
Added globalAlpha to the list of attributes copied for save/restore, as per the canvas specification.
Wow. Christian Effenberger pointed us to his impressive Snapfit.js project.
Snapfit takes any picture and, if the browser has support for canvas (or VML), will make it into a puzzle. Not just any puzzle, mind. A potentially wicked hard puzzle where the pieces can be mirrored in either axis. But, not to worry, if you need the puzzle solved, it will do an animated transition showing where the pieces should go, etc.
Christian sent us a few notes about performance, etc.:
The speed depends generally on picture dimension and number of puzzle pieces and particularly on browsers Javascript-Canvas-Performance.
It seems that Opera 9.x and Firefox 1.5 are to slow to be practicable, at least on my system (G4/1.25GHz).
Internet Explorer implementation in VML do not support all features and is unable to provide sub pixel precision. So don't expect to much.
It works in all the major browsers - Mozilla Firefox 1.5+, Opera 9+, IE6+ and Safari.
On older browsers, it'll degrade and your visitors won't notice a thing.
There is an interesting blog post* and demo code on Azarask explaining Flash Canvas as a solution for IE's lack of support for Canvas:
How does FlashCanvas work?
FlashCanvas is modeled after ExplorerCanvas which means it is a turn-key solution for adding Canvas support to IE. You can code away, happily using open standards and then use FlashCanvas to forcefully and silently upgrade IE to also being standards compliant.
There are two main components in FlashCanvas: the base FlashCanvas.swf flash file (a mere 688 bytes), and the FlashCanvas.js wrapper. I’ve used the excellent swfobject.js to embed the Flash into the page.
The FlashCanvas.js file implements a fake-canvas object and converts all existing canvas element into a flash object. The javascript intercepts canvas commands and forwards them to the FlashCanvas.swf movie file using the ExternalInterface provided by the flash player. The flash movie clip then interprets the command and draws accordingly.
Aza originally thought that this would also solve the performance problems other fallbacks to VML have but found out that sadly enough this is not the case because of the lag between ActionScript and JavaScript via ExternalInterface.
Each call via the ExternalInterface is taking approximately 0.5 ms. Since we know the time it takes to call an actionscript method from javascript, we can deduce the amount of time it takes for Flash to render. For the example1.htm page it takes ~24 ms to render 20 anti-aliased lines. In Safari a call into actionscript takes roughly 0.4 ms — this is only the call time, it doesn’t include the time to render anything. For example1.htm a single “particle” calls actionscript 3 times with the commands: [lineStyle,moveTo,lineTo]; each particle takes 1.2 ms in JS-to-AS calls; to take 1 second to render the frame ~833 particles need to be rendered.
There are few more ideas in the comments to the article and this might be an interesting concept to take further. Somehow it seems ironic to go back to Flash to make IE render standards, but then again VML is proprietary, too.
* Hat tip to Scott Schiller for pointing out this older, but somehow forgotten post.
.
