You searched for: 'native'

Friday, March 6th, 2009

Fun Anecdote about innerHTML

Category: IE

Just when you started having fun with innerHTML, you discovered that on IE, various table-related elements can’t be modified with it. Ever wonder why? Eric Vasilik is the developer responsible for that “feature” of IE, and he came clean about it several years ago:

We never had enough time to implement such a feature. And, in order to deal with attempts to modify tables in such a manner, I prohibited the modification of tables with innerHTML and other methods.

An alternative to all this would have been to “hack” something up. For example, I could have checked to see if the innerHTML of a TBODY was being set to something which began with a “

“. Under these circumstances I could have prepended a “

” to the string, and then plucked the TR’s out of the resulting tree and replaced the contents of the TBODY with them.

Sounds simple enough until you have to consider all the variations. Like, what if the string to be parsed looks like “

…”. Pretty soon you start doing all the work the real parser has to do.

So instead of hacking up something very incomplete and possibly erroneous in many cases, I left the modifications of tables with innerHTML out of the product.

The full blog entry goes into a bit more detail and is fairly interesting. Also, be sure to check out the post on easter eggs in early versions of IE.

Posted by Ben Galbraith at 8:00 am

3.3 rating from 27 votes

Monday, March 2nd, 2009

Manipulating video in real time with Canvas

Category: Canvas

Now that we are getting browsers that natively grok video as well as images and the like, we can do interesting things.

Paul Rouget has demonstrated “chroma-keying” (green screen effect) using JavaScript, the video tag, and canvas:

  1. The document establishes two canvas elements, with the IDs c1 and c2. Canvas c1 is used to display the current frame of the original video, while c2 is used to display the video after performing the chroma-keying effect; c2 is preloaded with the still image that will be used to replace the green background in the video.
  2. The JavaScript code is imported from a script named main.js; this script uses JavaScript 1.8 features, so that version is specified in line 22 when importing the script.
  3. When the document loads, the processor.doLoad() method in main.js gets run.

    The work to do the image manipulation is:

    computeFrame: function() {
    this.ctx1.drawImage(, 0, 0, this.width, this.height);
    let frame = this.ctx1.getImageData(0, 0, this.width, this.height);
    let l = / 4;

    for (let i = 0; i < l; i++) { let r =[i * 4 + 0]; let g =[i * 4 + 1]; let b =[i * 4 + 2]; if (g > 100 && r > 100 && b < 43)[i * 4 + 3] = 0; } this.ctx2.putImageData(frame, 0, 0); return; } [/javascript] Fun stuff!

Posted by Dion Almaer at 12:44 pm

4.2 rating from 29 votes

Friday, February 27th, 2009

TraceVis: Getting some eyes into TraceMonkey

Category: JavaScript

David Mandelin has announcedTraceVis “a visualization of TraceMonkey performance, with the goal of revealing what the JS VM is doing, and why it runs certain programs fast or slow, so we can figure out how to make the slow ones fast too.”

David starts off explaining what TraceMonkey is about, and pieces together the activities:

  • When TM starts running a program, it always starts by interpreting the program, exactly as in the non-tracing JS engine.
  • When execution reaches a point where TM might want to start a compiled trace, TM spends a bit of time monitoring the execution: checking to see if it already has a compiled region, counting the number of times passed, and deciding whether to start a trace. Monitoring is a kind of overhead: while monitoring, TM isn’t running the user’s program, but monitoring is a necessary cost of finding and optimizing traces.
  • If TM does decide to create a new compiled trace, it runs in the interpreter while recording the trace, including operations and types of values. During this time, it is running user code a little slower than the basic interpreter.
  • When the trace is finished, TM compiles the trace to native code. This is another form of overhead.
  • As I mentioned above, as part of monitoring, TM checks to see if it already has a compiled native trace starting at the current point. If so, TM selects the right trace and prepares to run it, which I call executing the trace. This is a third form of overhead.
  • Finally, TM can be running native code compiled previously. Compiled native traces run 2-20x faster than the interpreter, with a typical speedup factor of about 2.5.

He then discusses the visualization tactics, and what you get to see:

With these new eyes, David walks through various benchmarks to see how they are effected. Now that it is easier to see what is going on, the team should be able to optimize even faster.

Posted by Dion Almaer at 6:55 am
1 Comment

4.4 rating from 27 votes

Thursday, February 26th, 2009

Namespace.js: take a wild guess….

Category: JavaScript, Library

Let’s lead with code:


  1. Namespace('');
  2. = function() {};
  4. Namespace('com.example', {
  5.    MyClass: function() { return {}; }
  6. });
  7. var obj = new com.example.MyClass();
  9. Namespace.use('com.example.MyClass');
  10. var obj2 = new MyClass();
  12. // include com/example/RemoteClass.js
  13. Namespace.include('com.example.RemoteClass');
  14. var obj3 = new com.example.RemoteClass();
  16. Namespace.registerNativeExtensions();
  17. ''.namespace();

Reading the code above is probably all you need to know about the new, library independant, JavaScript library Namespace.js, created by Maxime Bouroumeau-Fuseau.

The goals for the library are:

  • Simple API
  • Framework independent
  • Remote file loading (synchronously or async)
  • Tested against Firefox 3.x, Safari 3.x, IE 6/7 and Opera 9.5
  • Highly configurable
  • Events
  • Optionally add methods to native objects

You can see more in the How To.

Posted by Dion Almaer at 2:10 pm
Comment here

3.2 rating from 29 votes

JSONView: JSON browser from within Firefox

Category: JSON, Utility

JSONView is a new Firefox extension that gives you a nice way to view your JSON documents (JSONovich also does the trick).

Ben Hollis talks about the extension:

The extension itself is pretty simple. I wasn’t sure how to approach the problem of supporting a new content type for Firefox, so I followed the example of the wmlbrowser extension and implemented a custom nsIStreamConverter. What this means is that I created a new component that tells Firefox “I know how to translate documents of type application/json into HTML”. And that it does – parsing the JSON using the new native JSON support in Firefox 3 (for speed and security) and then constructing an HTML document that it passes along the chain. This seems to work pretty well, though there are some problems – some parts of Firefox forget the original type of the document and treat it as HTML, so “View Page Info” reports “text/html” instead of “application/json”, “Save as…” saves the generated HTML, Firebug sees the generated HTML, etc. Just recently I came across the nsIURLContentListener interface, which might offer a better way of implementing JSONView, but I’m honestly not sure – the Mozilla documentation is pretty sparse and it was hard enough to get as far as I did. I’m hoping some Mozilla gurus can give me some pointers now that it’s out in the open.

Posted by Dion Almaer at 10:31 am

4.1 rating from 17 votes

Wednesday, February 25th, 2009

Titanium PR2 Released

Category: Runtime, Titanium

A little over two months after their initial launch of Titanium, Appcelerator today followed with the “PR2” release of their open-source web-as-desktop-app run-time platform (i.e., an open-source competitor to Adobe AIR).

This release adds support for Linux to their OS X and Windows versions and also provides a GUI for creating stand-alone Titanium apps (the previous version relied on command-line utilities). This application is a Titanium app itself and includes some innovative features, including a view of the Titanium group’s Twitter / FriendFeed streams and a direct connection to their IRC channel.

Appcelerator is also trying to make it as easy as possible hack away on the Titanium platform without requiring the use of C++; they’ve added a new module API that lets you use JavaScript, Ruby or Python in addition to Bjarne Stroustrup‘s spawn.

On a lower level, and of more probable interest to our community, the JavaScript APIs have been greatly extended. Let’s consider the File API in the first release of Titanium, which consisted of one documented method (“read”) and one undocumented method (“write”). In the PR2 release, it looks quite a bit richer:

copy ( to ) : boolean
    Copies a file to another location.
createTimestamp ( ) : number
    Returns a timestamp of the file’s creation.
createDirectory ( recursive ) : boolean
    Creates a directory.
deleteDirectory ( recursive ) : boolean
    Deletes an existing directory.
deleteFile ( ) : boolean
    Deletes a file.
exists ( ) : boolean
    Checks whether or not a file exists on the user’s system.
extension ( ) : string
    Returns the file’s extension.
getDirectoryListing ( ) : array
    Returns an array of files inside a directory.
isDirectory ( ) : boolean
    Checks whether the file object is a directory.
isFile ( ) : boolean
    Checks whether the file object is a file.
isHidden ( ) : boolean
    Checks whether the file object is hidden.
isSymbolicLink ( ) : boolean
    Checks whether the file object is a symbolic link.
modificationTimestamp ( ) : string
    Returns a timestamp of the file’s last modification.
move ( to ) : boolean
    Moves a file to another location.
name ( ) : string
    Returns the name of the file.
nativePath ( ) : string
    Returns the full path of the file.
parent ( ) : File
    Returns the parent directory where the file resides.
read ( ) : string
    Reads the content of the file.
readLine ( reset ) : string
    Reads a particular line in the file’s content.
resolve ( path ) : boolean
    Changes the file or directory referenced by a File object.
size ( ) : number
    Returns the size of the file.
spaceAvailable ( ) : number
    Returns to space available on a user’s system.
toString ( ) : string
    Returns the file’s properties as a string.
write ( data [, append] ) : boolean
    Outputs data into a file. 

Check out the full API docs to get a feel for the rest of the APIs; but being an open-source project, Titanium’s newest APIs are still in the source only.

In terms of the run-time itself, they’ve updated their WebKit renderer to be very close to the latest and greatest and it includes all of the new HTML 5 goodies that Safari 4 has.

Nolan Wright, Appcelerator CTO, has created some screencasts to show off some of the new features; here’s one of them:

We’re very excited to see Titanium moving along!

Posted by Ben Galbraith at 6:01 am

4.4 rating from 34 votes

Ext JS and Adobe AIR

Category: Adobe, Bespin, JavaScript, Sencha

As we’ve been exploring ways to take Bespin to the desktop, we’ve looked closely at some of the single-site browser technologies and their individual APIs to work out which platform we favor. As we went through this exercise, Adobe AIR really impressed us with the richness of its JavaScript APIs which wrap native APIs (as well as the accompanying documentation).

There’s no other competing JavaScript API that we’ve seen that comes close to this level of comprehensiveness (please correct us in the comments in we’re wrong).

We weren’t able to use AIR as its implementation of <canvas> is a bit old, but we’re excited to play with it as soon as we can.

Ext JS and Pixel Bender Explorer

We’re not the only ones who have been playing with AIR. Aaron Conran from the Ext JS crew blogged about Pixel Bender Explorer, an AIR app they wrote to demonstrate how to integrate Ext JS with AIR’s Pixel Bender technology that lets you apply pretty amazing effects to interfaces.

Pixel Bender can spruce up an Ext.air application by adding custom animations to wow your users. However, you should be cautious about the over-use of these filters throughout your application. For a good example of how effective these filters and animations can be to provide proper user feedback you should check Adobe’s signature sample BlackBookSafe. Each time an animation occurs it is clear why it happened, not a surprise to the user and adds character to the application. When using these animations you should strive for the same goal, not to surprise your user, but to impress them.

Check out the AIR application to see some pretty amazing fx.

Posted by Ben Galbraith at 5:00 am

4.4 rating from 57 votes

Tuesday, February 24th, 2009

Big News from Cappuccino: Aristo and Atlas

Category: Cappuccino, Framework, Toolkit

As it turns out, the Cappuccino team has been busy hacking away on some very impressive stuff. Today at the Future of Web Apps Miami, they announced Aristo and Atlas.


Cappuccino worked with the popular design firm Sofa to create Aristo, a new open-source look-and-feel that will be freely available, including the source PSD files complete with all layers intact. Anyone may use the look in their own projects. I didn’t get a close look at the theme, but it seemed quite nice.


Atlas continues Cappuccino’s efforts to essentially port Apple’s Cocoa stack to the Web. Atlas itself looks like a mash-up of Flex Builder and Interface Builder, all running in a browser, but continues to hew very closely to Interface Builder’s approach. For example, just as with Interface Builder, you can bind properties on components to other components or to code through visual drag-and-drop operations (complete with the same blue connector line).

What’s more, Atlas introduces the equivalent of Interface Builder’s “nib” file (though I forget what their version is called).

In a live demo, Francisco Tolmasky built an RSS reader using an entirely visual approach. Francisco made it clear that he doesn’t think code will go away, but instead, he feels the amount of code required to create a web application will dramatically decrease.

They also released a screencast:

Multi-platform Support, including iPhone JavaScript Wrappers

Atlas also supports targeting multiple platforms, letting you create different interface files that bind to the same back-end code. They’ve even bridged native APIs and exposed them to the JavaScript environment and provide a PhoneGap-like way to run such applications as native iPhone applications. Wow.

It’s hard to overstate how impressive the Atlas demo was; the release should be coming at some point in summer 2009.

Great job guys!

Posted by Ben Galbraith at 5:20 pm

4.2 rating from 56 votes

Cufón – custom font embedding with Canvas/VML

Category: SVG

Probably the most requested CSS feature of designers is being able to use custom fonts on web sites. Right now the only real way of doing that cross-browser is relying on Flash, either by building the whole page in it or by using the “Scalable Inman Flash Replacement” or short SIFR script. This does the job for most people but Simo Kinnunen wanted to avoid having to use proprietary software and go for canvas instead to reach the same goal.

cufon generator

The goals for Cufón were:

  • Absolutely no plugins required – Everything needed to display Cufón is already available by default in your visitor’s browser. Rather than requiring flash to make the switch, Cufón relies simply on javascript alone.
  • Compatibility – Cufón runs on IE6, 7, 8, Firefox and the latest version of Safari. On other unsupported browsers (IE5-, Safari 2) it will fail gracefully.
  • Ease of use – Little or no configuration required.
  • Speed – Cufón is quick. It loads almost instantly with no ‘flicker’ that you would normally experience with sIFR.

Technically Cufón is a web interface to fontforge and creates an SVG font on the fly from your source font using a JavaScript renderer.

Chris Jennings has more details on it and there’s a handy online generator for converting your fonts.

Visually this is pretty cool, however now we need to give it the same rigorous poking we gave SIFR over the years: what does it do in terms of accessibility, usability (copy and pasting text), scaling of text and most importantly overall page performance?

The other problem of course is copyright of font faces, and this is a much harder issue to tackle. Another solution with the same approach, typeface.js had issues with that in the past and this will not be different.

Got any test data?

Posted by Chris Heilmann at 8:53 am

3.8 rating from 33 votes

Friday, February 20th, 2009

Comparison of JavaScript Inheritance Mechanisms; Proposal

Category: JavaScript

Andrea Giammarchi took the time to compare the JavaScript inheritance mechanisms in base2, Dojo, jsClass, MooTools, Prototype, and YUI and illustrates the same scenario with all of those frameworks. Handy indeed!

He concludes with a proposal:

WebReflection Proposal
After an analysis like this one, how could I skip my “all the best from others without troubles” proposal?
These are my considerations:

* The fastest way to use a parent method is an explicit call

* The best way to perform above step is via a link able to remove explicit dependencies (read: MyParentClassName instead of this.constructor.superclass). In this way methods could be easily transported from a prototype to another one without problems (less memory usage and less code to maintain)

* The this.parent() solution is the cleanest one and generally speaking more close to our concept of “Object Oriented JavaScript”. It is portable, it is meaningful, it is shorter than an explicit call with or without a link but it is not that fast to execute.

* Using best practices to cache all we need, create a quick wrapper, resolving hidden methods problems, will not bring us to native or YUI performances, but at least in a good scenario: fast enough to guarantee performances over code elegance and readability

So here we are with my proposal:

* Concept: parent in prototype but changed runtime for hierarchy purpose only if necessary (read: only for overrides). This will allows us to call parent directly in the constructor, as example, so we won’t have double calls for each instance. At the same time, this implementation lets us call explicitly a parent method from those whose were not inherited.

* Troubles: performances are the best but still far from native one.

Here’s the usage of his proposed library:

  1. function WRClass(name){
  2. = name;
  3. };
  4. WRClass.prototype.toString = function(){
  5.     return;
  6. };
  7. WRClass.prototype.toLocaleString = function(){
  8.     return "locale: " + this.toString();
  9. };
  10. function WRSub(name, age){
  11.     this.parent(name);
  12.     this.age = age;
  13. };
  14. wr.extend(WRSub, WRClass, {
  15.     toString:function(){
  16.         return this.parent() + ": " + this.age;
  17.     }
  18. });
  19. var me = new WRSub("Andrea", 30);
  20. alert(me);                   // Andrea: 30
  21. alert(me.toLocaleString());  // locale: Andrea: 30
  23. // last, but not least
  24. WRSub.prototype.explicitParentToString = function(){
  25.     return;
  26.     // instead of
  27.     return;
  28. };
  29. alert(me.explicitParentToString()); // Andrea

And, see the blog post linked above for the source code to his implementation.

Posted by Ben Galbraith at 8:00 am

2.9 rating from 41 votes

Wednesday, February 18th, 2009

Canvas for a Text Editor?

Category: Bespin, Canvas

The following is a lightly edited repost from an entry on my personal blog.

Culling through the Bespin feedback we received post-launch, it’s clear that a few aspects of the project are controversial. I’d like to address one of them: our choice of canvas as the rendering engine for the editor.

(In addition to feedback, we’ve already seen some interesting community efforts at extending Bespin; check out our blog post on the Labs website for details.)

Bespin and Canvas

Bespin takes advantage of a relatively new feature of browsers to power its text editor: the <canvas> element. Canvas gives developers something the intriguing ability to render arbitrary two-dimensional graphics in a web page; it’s basically an <img> element you can draw into with JavaScript commands. The API looks something like this:

  1. // all drawing performed via the context
  2. var ctx = document.getElementById("canvasId").getContext("2d");
  4. ctx.fillStyle = "black";  // accepts same values as CSS "background" property
  6. ctx.fillRect(0, 0, 50, 50); // draws a black 50x50 rectange in the canvas

Canvas is available in the currently released versions of Safari, Opera, Firefox, and Chrome; it has been a part of the HTML 5 draft specification for a long time. It is not available in Internet Explorer.

Bespin builds on this API to recreate a text editor from scratch. That’s right; rather than use the native facilities of every modern operating system that give you word processing features for free, Bespin rolls them from scratch, including such basic constructs as a blinking cursor and text selection.

One of the the features we didn’t have to re-implement is text rendering; the HTML 5 specification includes an API to render text on a canvas. Unfortunately, this portion of the canvas API has only been implemented by Firefox, though it’s already in WebKit nightly builds (and thus will very likely be in a future version of Safari) and Chromium nightlies (from which Chrome versions are derived). We expect Opera will soon as well.

But Does It Scale?

We weren’t actually sure this would work when we set out to do it. You wouldn’t think a JavaScript interpreter in a browser could render enough frames on a large, modern 32-bit image display to make the experience anything close to what you get effortlessly in Notepad on Windows XP running on 10-year-old hardware.

To be sure, for some users, the performance does indeed suck–we’re looking into exactly why (it appears that certain video drivers cause performance problems on Linux; Google Chrome’s multi-process architecture appears to slow down performance on that browser). The interesting thing is that for large classes of users, performance is really, really good. There’s a lot to say about topic; we’ll touch on that more in a future blog post.


The first time we demoed Bespin to someone who understand these implementation details, the reaction was “Wow, you’re crazy!” followed by “That’s really cool.” Pretty much everyone else I’ve spoken with has the same first reaction (the crazy bit) but the follow-up is not always positive (ranging from the aforementioned to “WTF!?”).

Surely there’s a higher-level abstraction we could have built on? Something above pixel painting?

Well, let’s get one thing out of the way first-up: there may very well be a more effective way to do what we’ve done than with canvas. I’m not saying this is the best way to do it; I just want to explain why we did it.

Control, Control, We Must Have Control

One of our goals with Bespin is to be able to (at some point in the future) deliver a fantastically compelling user experience for editing code on the Web. We don’t want Bespin to be the editor you use when you can’t get to your desktop; we want it to be your editor of choice in any context.

It turns out that’s a tall order. Some users like word-wrapping; some don’t. Some users actually like column selection modes while other (more sane) users like line-oriented selection (I kid, I kid). Developers are notoriously finicky with their tools–especially their code editor–and we didn’t want to re-educate the world on how our editor worked; we want to make Bespin adaptable to as wide an audience as we could.

We felt that if we built Bespin on something like <textarea> or the “content editable” feature of HTML, we’d have a built-in set of limitations over which we’d have very little or no control.

The Alternatives

To build on the last paragraph, we considered three different alternatives to canvas:

  1. textarea
  2. contentEditable
  3. “line blocks”

The “control” argument above is one reason we dismissed textarea and contentEditable; further, we’ve seen a large number of other web editor implementations based on these approaches and based on playing with those and by the many horror stories floating around the Web from their authors, we were concerned that we couldn’t overcome performance and compatibility problems, especially when editing large files.

Line Blocks

“Line blocks” are a different story. What if we had a bunch of <div> elements stacked vertically and treated each one as a rendering surface for a line of text in the document being edited? We played with that approach a bit but couldn’t get as much performance out of it as we could with canvas, especially for wide files (>80 columns).


We’ve taken some flak for claiming that Bespin is based on “Open Web technologies” but only supporting Firefox and WebKit nightly. Earlier in this post, I explained the state of canvas support in the browsers and how we’re hopeful that our canvas approach will work in future versions of Chrome, Safari, and Opera.

However, we have a plan in case browsers don’t implement the proper text rendering APIs: bitmap font rendering. By pre-rendering fonts into images, we can then render the images on the canvas. There are a number of proof-of-concept implementations floating around that demonstrate this concept (though unlike some, we wouldn’t render the fonts as vectors–there’s no way it would perform to our needs).

What about Internet Explorer? It’s hard to say what the future of canvas support is for IE; it certainly isn’t coming in IE8. However, there are a number of projects that emulate canvas by mapping it to VML (an abandoned IE proprietary rendering technology), Flash, and Silverlight. So far, none of these bridges offers adequate performance for us to seriously considering supporting it. We’re aware of promising developments in this area and we’ll experiment with IE support as these technologies evolve.

Mobile Support: iPhone, Android, Fennec, and More

We’re also very excited to get Bespin working on mobile devices with fully-featured browsers. The iPhone’s browser–”mobile Safari”–does support canvas, but does not yet have the text rendering APIs. We may implement the aforementioned bitmap font rendering just to get on the iPhone. We are also looking at other mobile platforms, such as Android, Fennec, Palm, and so forth.

How cool would it be to change your code and redeploy it from the beach?

Obviously, it will take some interface changes for Bespin to be usable on a mobile device. We have some ideas here and we look forward to working with the community on developing the concepts further.

What do you think?

What do you think of our use of canvas in Bespin’s editor?

Posted by Ben Galbraith at 8:00 am

4 rating from 50 votes

Tuesday, February 17th, 2009

webOS and Mojo framework for Palm Pre; Finally get some info

Category: Mobile

Fancy writting a Web application on the architecture above? Since announcing the Palm Pre we have seen a lot of people wondering about the goods under the hood. We now have an online reference to check out.

Once you get past the high level stuff, you get to the meat:

Stages and Scenes

Palm’s user experience architecture provides for a greater degree of application scope than is normally considered in a web application. To support this and specific functions of the framework, Palm has introduced a structure for webOS applications built around stages and scenes.

A stage is a declarative HTML structure similar to a conventional HTML window or browser tab. Palm webOS applications can have one or more stages, but typically the primary stage will correspond to the application’s card. Other stages might include a dashboard, or other cards associated with different activities within the application. You should refer back to the earlier example of the Email application where the main card held the Email inbox and another card held a draft Email. Each email card is a separate stage, but part of the same application.

Scenes are mutually exclusive views of the application within a Stage. Most applications will provide a number of different kinds of scenes within the same stage, but some very simple applications (such as Calculator) will have just a single scene. An application must have at least one scene, supported by a controller, a JavaScript object referred to as a scene assistant, and a scene view, a segment of HTML representing the layout of the scene.

Application Lifecycle

Palm webOS applications are required to use directory and file structure conventions to enable the framework to run the applications without complex configuration files. At the top level the application must have an appinfo.json object, providing the framework with the essential information needed to install and load the app. In addition, all applications will have an index.html file, an icon.png for application’s Launcher icon, and an app folder, which provides a directory structure for assistants and views.

By convention, if the app has images, other javascript or application-specific CSS, then these should be contained in folders named images, javascripts, and stylesheets respectively. This is not required but makes it simpler to understand the application’s structure.

Launching a webOS application starts with loading the index.html file and any referenced stylesheets and javascript files, as would be done with any web application or web page. However, the framework intervenes after the loading operations and invokes the stage and scene assistants to perform the application’s setup functions and to activate the first scene. From this point, the application would be driven either by user actions or dynamic data.


  • Storage: HTML5 storage is natively accessible
  • Widgets: Can look like this:
    1. <div x-mojo-element="ToggleButton" id="my-toggle"></div>
  • Services: The architecture looks like this:
  • Developer Tools: SDK comes with emulator and command line tools

Still looking forward to more details, examples, and such…. but exciting to see the data coming out!

Posted by Dion Almaer at 12:53 am

3.7 rating from 23 votes

Monday, February 16th, 2009

Designing a JavaScript client for a REST API

Category: Articles, JavaScript

This is a guest post by Jared Jacobs of the KaChing, an exciting new way to do your own hedge fund, the Web 2.0 way (a.k.a. don’t give it to Madoff!). I was very happy when he said he would be willing to do a post on REST APIs, and what makes a good design.

So you want to write a script that sites all over the web can use to access your REST API, eh? Well, that would be pretty straightforward if it weren’t for two things:

  1. browser same-origin restrictions on XMLHttpRequests (XHRs) and inter-window/frame access
  2. the lack of wide browser support for HTML 5-style message passing between windows.

The Same-Origin Policy and its minor browser-specific variations are detailed elsewhere, so I’ll just summarize it with a few key points. I use the term window to mean window object, which can be a top-level page or reside inside a frame or iframe.

  • XHRs can only be issued to the same domain as the originating window (not the originating script).
  • Two windows can read, write, and call each other’s properties if and only if they are from the same domain (host, port, and scheme).
  • There is one notable exception to the previous statement that is consistent across browsers: A window can change (but not necessarily read) the location of its own iframes and of the top window, regardless of their domains.

Recall that native browser primitives for sending or receiving information across domains include frames, iframes, images, scripts, stylesheets, and forms.

With these constraints and possibilities in mind, let’s first consider how best to support GET requests to the REST API.

Transport for GET Requests

XHRs would be the ideal way to retrieve data, since they provide meta information about each response, such as the the HTTP status code, in addition to the content. As noted above, however, other sites cannot use XHRs to directly to access your REST API.

One possible workaround is for each site that uses the JS client to install a server-side proxy on its domain that proxies REST API calls to your servers. That’s a pretty big barrier to the adoption of your JS client, though. We can do better.

Another possibility is having the JS client delegate the XHRs to a helper iframe that it creates specifically for that purpose. The iframe’s page must, of course, reside on the same domain as the REST API. Then there are also the problems of 1) passing the request URI from the parent window to the helper iframe and 2) passing the response status and content from the helper iframe back to the parent window. These problems are manageable, but not worth solving for GET requests, it turns out. We’ll return to them later.

Textual data can also be loaded across domains within scripts and stylesheets, both of which are lighter-weight than iframes and free of same-origin restrictions. To use one of these techniques, you must 1) either embed the data in a comment or encode it as part of either a valid script or stylesheet, and 2) determine when the data has finished loading.

A popular solution that addresses these two issues nicely, sometimes called script transport or JSONP, is formatting the data in JavaScript Object Notation (JSON) and wrapping it in a JS function call. This is the route I chose for kaChing’s JS client. A JSONP request typically includes the callback name as an HTTP request parameter. You’ll want to host a simple proxy on your domain that handles these requests by taking off the callback name parameter, relaying the request to your REST API servers, then formatting the response as the appropriate JS function call. Your JS client should generate a unique callback name for each request to avoid mix-ups in the event that multiple requests overlap in time. Also make sure that your proxy can relay both success and error responses from the REST API to your JS client.

Another nice property of the JSONP approach is that anyone can write a JS client and run a JSONP proxy server for any REST API, then share their JS client with anyone who trusts them.

Two weaknesses of the JSONP approach to be aware of: 1) it adds another failure point (the proxy server), and 2) detecting loading errors is harder with scripts than with XHRs.

Transport for Non-GET Requests

For HTTP methods other than GET (e.g. HEAD, POST, PUT, DELETE), script transport would require disguising the request as a GET, which has different semantics. Then there’s also the fact that a large request might not fit into a single URL. There must be a better alternative.

HTML forms can do POSTs, but that still leaves out the other HTTP methods.

What about the approach we proposed earlier—delegating XHRs to helper iframes? The JS client can construct a hidden iframe each time it needs to make a non-GET request. Iframe construction is quite fast (7-25 ms for most browsers on today’s machines) if the iframe page is small and cached. The request specification (i.e. verb, URL, and possibly parameters and content) can be passed to the helper iframe in its URL fragment.[1] The helper iframe page deserializes the request specification from its URL fragment immediately upon loading and issues the request. Simple enough. Now to relay the server’s response to the parent window. The simplest and most efficient way is HTML 5’s window.postMessage. The latest versions of some browsers support it already, but most people’s browsers don’t yet. (IE 6 & 7 don’t, but IE 8 will.) Unfortunately, the best two alternatives[2] to window.postMessage both require additional cooperation from sites using the JS client.

Approach 1. Sites using the JS client must host a small HTML file that we’ll call caller.html. They should inform your JS client of its URL as an initialization step. When the XHR response arrives, the XHR iframe creates a child iframe pointing to caller.html and including a URL fragment containing the response status, content, and a callback name from the original window. caller.html simply parses its URL fragment and passes the response information to its grandparent window via a direct function call that looks something like this: parent.parent[callbackName](status, responseText).

Approach 2. Sites using the JS client must inform your JS client of the URL of any valid resource on their domain as an initialization step.[3] That’s right, it can be an image, a stylesheet, an HTML page, or anything else—anything they don’t mind you requesting repeatedly. Its content is irrelevant. It just has to exist. We’ll call it cleardot.gif. When the XHR response arrives, the XHR iframe creates a child iframe pointing to cleardot.gif and including a URL fragment containing the response status and content. The JS client will have been polling for the existence of this grandchild iframe and can decode the response from its location’s URL fragment.

For kaChing’s JS client, I chose Approach 1 because it avoids polling and feels less hacky. Asking sites to allow our JS client to reuse an existing resource on their site seems more fragile because the resource’s original purpose may change or disappear someday. I also built in future compatibility with window.postMessage. (For our implementation, see the _send method in kaChing’s client.js. The helper iframe pages are xhr.html and caller.html.)

Regardless of which approach you choose, it’s important for optimal performance that the helper files (both the ones that you are hosting and the ones that your clients are hosting) be long-term cacheable. Also consider having your JS client preload them to warm the browser cache.

The Programming Interface

Now that we’ve decided on transport mechanisms, let’s turn our attention to the JavaScript programming interface. Since our transport mechanisms are all asynchronous, callers who care about the results of their requests will need to provide callbacks.

A first stab might be to have a JS method for each (verb, resource) pair in the REST API:

kaching.getUser(31, function(user) {

  // Use user.

That approach is certainly clear and simple, and is probably the best pattern for requests with side effects, like POST, PUT, and DELETE.

An improvement for GET requests would be what I call the shopping list pattern, which allows callers to easily request multiple resources together with a single callback:


  function(user, portfolio, watchlist) {
    // Use user, portfolio, and watchlist.

In addition to the added convenience, this pattern reduces latency by allowing your JS client to request the desired resources in parallel. The two tricky parts in the implementation of this pattern are what I call the joining callback, which collects all of the expected responses before firing, and the use of Function.prototype.apply to invoke the caller’s callback. For a working example of this pattern, see kaching.fetch in kaChing’s client.js.

Other best practices for JavaScript libraries to consider:

  • Avoid polluting the global namespace. Define only a single global name (e.g. kaching).
  • Make the loading of your library idempotent, in case it gets included in a page multiple times.
var kaching = kaching || ...;


Authentication issues are beyond the scope of this post. If you’re looking for more information on this topic, you might consider starting with the Google Data APIs Authentication Overview.

Your Prize

For making it this far, you’ve earned an invitation to:

The API Garage Event

1:00pm Sat Feb 21 at kaChing HQ

Come eat, drink, celebrate, meet the kaChing team, and hack on our API.

Oh, and we’re hiring!


[1] We brush up against a limit on request size here—namely, the browser’s upper bound on URL length—but it’s not insurmountable. If you anticipate large requests, you can create additional helper iframes that each accept a portion of the request data in their URL fragments along with a sequence number and then pass them to the XHR iframe via a direct function call that looks something like this: parent.frames[xhrFrameName].acceptRequestData(data, seqNumber).

[2] Here I thought I’d catalog a few other ways of relaying the server’s response to the parent window that I considered or tried and why I nixed them.

Attempt Outcome
When the response arrives, set the parent window’s location fragment. Doing this adds an annoying new browser history entry. Plus, the new location fragment is visible in the browser’s location bar.
Before issuing the request, create a helper iframe with no src attribute in the parent window; then when the response arrives, set its location to that of the parent window plus a fragment. In IE 7 this works, but it results in the browser loading the containing page in the helper iframe, a potentially large waste of resources. Permissions errors in Firefox and Safari.
Before issuing the request, create a helper iframe with src=”about:blank” in the parent window; then when the response arrives, add a url fragment. Permissions errors again while adding the fragment in Firefox and Safari. Permissions error when the parent tried to read the fragment in IE 7.
Before issuing the request, create a helper iframe with src=”/favicon.ico” in the parent window; then when the response arrives, add a url fragment. This works in IE for sites that have favicons, but it’s not a very robust solution. Plus, if the site’s favicon isn’t very cacheable, your JS client could end up re-downloading it with each API call.
Instead of adding the xhr iframe directly to the page, create an intermediate iframe with no src attribute and document.write its content (just the xhr iframe); when the response arrives, the xhr iframe sets the url fragment of its parent, the intermediate iframe. I thought this might work because the intermediate iframe inherits the document.domain of its parent, so the parent should be able to poll its location. The problem I encountered was that the intermediate iframe’s window had no location property to set (presumably since that iframe had no src attribute).

[3] A variation on Approach 2 is not to ask sites to inform your JS client of the URL of a cacheable resource, but instead have your JS client probe for a favicon and/or walk the DOM of its host page to identify a suitable image or stylesheet. While this could work in many cases, I’d advocate being up front about the requirements of your JS client and avoiding making unnecessary assumptions.

Posted by Dion Almaer at 9:01 am

3.5 rating from 28 votes

Thursday, February 5th, 2009

Opera comes out with new JavaScript engine in Carakan, new hardware API in Vega

Category: JavaScript, Opera, Performance

Opera isn’t sitting on their heels as the other browser vendors get snappy (even if other claim so!)

Today they announced Carakan a new register based JavaScript VM that is currently 2.5 times faster than their existing one (based on SunSpider). It does native code generation including at specializing for Regex (interestingly since irregex for Chrome was just announced too). Here is their words:

We have focused our efforts to improve upon our previous engine in three main areas:

Register-based bytecode

The last couple of generations of Opera’s ECMAScript engine have used a stack-based bytecode instruction set. This type of instruction set is based around a stack of values, where most instructions “pop” input operands from the value stack, process them, and “push” the result back onto the value stack. Some instructions simply push values onto the value stack, and others rearrange the values on the stack. This gives compact bytecode programs and is easy to generate bytecode for.

In the new engine, we’ve instead opted for a register-based bytecode instruction set. In a register-based machine, instead of a dynamically sized stack of values, there’s a fixed size block of them, called “registers”. Instead of only looking at the values at the top of the stack, each instruction can access any register. Since there is no need to copy values to and from the top of the stack to work on them, fewer instructions need to be executed, and less data needs to be copied.

Native code generation

Although our new engine’s bytecode instruction set permits the implementation of a significantly faster bytecode execution engine, there is still significant overhead involved in executing simple ECMAScript code, such as loops performing integer arithmetics, in a bytecode interpreter. To get rid of this overhead we are implementing compilation of whole or parts of ECMAScript programs and functions into native code.

This native code compilation is based on static type analysis (with an internal type system that is richer than ECMAScript’s ordinary one) to eliminte unnecessary type-checks, speculative specialization (with regards to statically indeterminate types) where appropriate, and a relatively ambitious register allocator that allows generation of compact native code with as few unnecessary inter-register moves and memory accesses as possible.

Automatic object classification

Another area of major improvement over our current engine is in the representation of ECMAScript objects. In the new engine, each object is assigned a class that collects various information about the object, such as its prototype and the order and names of some or all of its properties. Class assignment is naturally very dynamic, since ECMAScript is a very dynamic language, but it is organized such that objects with the same prototype and the same set of properties are assigned the same class.

Nice to see, and interesting that the browsers aren’t (or aren’t able too?) share their VM work. Each browser has a new VM implementation going. Wouldn’t it be nice if they could share effort?

Then we have Vega a vector graphics library that Opera uses to power SVG, Canvas, and more.

Vega was created shortly after we started working on SVG support.
When we added SVG support in Opera we needed a vector graphics
library. We looked into what what available to use and met our
requirements (fast, low memory usage and works on platforms
ranging from phones to TVs and desktop computers). We did not
find and good match for our needs, so we decided to write our

Shortly after we created Vega we added <canvas> support, which
also uses Vega.

The most recent addition to Vega is the ability to use a
hardware accelerated back-end. The back-ends we are using at the
moment are OpenGL and Direct3D.

A busy day for Opera!

Posted by Dion Almaer at 12:01 am

4.5 rating from 35 votes

Wednesday, February 4th, 2009

irregexp: New fast Regexp engine in Chrome

Category: Chrome, Performance

We are seeing great work with faster JavaScript, but what about faster DOM? Or faster CSS? Or faster libraries? Or faster Regexp? Well, members of the Chrome team have announced irregexp, a faster Regexp engine for Chrome. They go into detail on what they had, what they did, and why they did it.

It is also well known that SunSpider tests Regexp that isn’t targeted to what is actually done on the Web. The V8 benchmark has updated tests that aim to do a better job here. Being Google, they also have this index laying around, so they took advantage of that to test against popular webpages:

During development we have tested Irregexp against one million of the most popular webpages to ensure that the new implementation stays compatible with our previous implementation and the web. We have also used this data to create a new benchmark which is included in version 3 of the V8 Benchmark Suite. We feel this is a good reflection of what is found on the web.

I would love to see this available for other browsers to test on :)

And now, here are the details on the changes:

While the V8 team has been working hard to improve JavaScript performance, one part of the language that we have so far not given much attention is regexps. Our previous implementation was based on the widely used PCRE library developed by Philip Hazel at the University of Cambridge. The version we used, known as JSCRE, was adapted and improved by the WebKit project for use with JavaScript. Using JSCRE gave us a regular expression implementation that was compatible with industry standards and has served us well. However, as we’ve improved other parts of the language, regexps started to stand out as being slower than the rest. We felt it should be possible to improve performance by integrating with our existing infrastructure rather than using an external library. The SquirrelFish team is following a similar approach with their JavaScript engine.

A fundamental decision we made early in the design of Irregexp was that we would be willing to spend extra time compiling a regular expression if that would make running it faster. During compilation Irregexp first converts a regexp into an intermediate automaton representation. This is in many ways the “natural” and most accessible representation and makes it much easier to analyze and optimize the regexp. For instance, when compiling /Sun|Mon/ the automaton representation lets us recognize that both alternatives have an ‘n’ as their third character. We can quickly scan the input until we find an ‘n’ and then start to match the regexp two characters earlier. Irregexp looks up to four characters ahead and matches up to four characters at a time.

After optimization we generate native machine code which uses backtracking to try different alternatives. Backtracking can be time-consuming so we use optimizations to avoid as much of it as we can. There are techniques to avoid backtracking altogether but the nature of regexps in JavaScript makes it difficult to apply them in our case, though it is something we may implement in the future.

Posted by Dion Almaer at 5:19 pm

4.4 rating from 26 votes

Wednesday, January 28th, 2009

Timescope: to mobile and beyond

Category: GWT, Showcase

Ray Cromwell has the ability to do amazing things in the browser. We have often linked to his work on Chronoscope and more, and this time he has a screencast showing of Timescope “our charting engine, written in GWT, runnable as Servlet or Android native application, renders huge numbers of points at interactive rates, scriptable, annotatable like Google Maps, and stylable via CSS-like stylesheet mechanism.”

For those who know of Chronoscope, this is the commercial version, big changes include more accurate multiresolution filtering algorithm, Android native version, Chart Server, iPhone integration, synthetic datasets, a number of other things. We eventually hope to include a subset of complete implementation of R in GWT, to allow statistical work to be done totally in browser.

At the 7:20 mark, you can see GSS, which is an implementation of a CSS parsing and cascade engine in GWT, to support our totally custom set of properties, elements, and pseudo-classes.

Very cool indeed Ray.

Posted by Dion Almaer at 8:08 am

2.7 rating from 64 votes