Editor’s note: Dan Kantor is the CEO behind the awesome ExtensionFM project. It really pushes the boundaries on what the Web can do, so I asked Dan to give us a mini case study on the project. What follows is his words on the matter. Thanks for taking the time Dan!
Dion recently posted about the march to a more client-centric web. The post hit home with me as the founder of a web-based application that is as much ‘app-like’ as it is ‘web-like’. The promise of the web, and HTML5 in particular, is that we will finally reach the write-once, run-anywhere dream we have all been in search of that neither Java nor Flash could fully deliver. Dion poses this question (and answer) towards the end of the post – ‘As a developer, do you want to port experiences between incredibly varied platforms such as Web, iPhone, Android, WinPho 7, RIM, Kindle SDK, [insert many others!]? No.’. I don’t think many developers would argue with Dion’s answer. The real question is – ‘Are we there yet?’
My company is developing a client-centric web-based music player that cuts no corners in its attempt to look, feel and run like a native application. We threw progressive enhancement out the door, pushed the pedal to the HTML5 metal and created something that most people would be hard-pressed to believe is not a native application if they didn’t already see it running inside the browser. However, there is a catch. Our application only runs inside Chrome. This was of course by design. We built it as a Chrome extension. We didn’t choose to do this because we wanted to restrict it to Chrome. We did this because a large part of the functionality of our application needs the extra APIs and permissions an extension gives you. A nice side-effect of that is we know 100% of our users are using an HTML5 capable browser. Chrome extensions are 100% HTML(5), CSS(3) and Javascript. Extensions provide extra Javascript APIs that give you access to functionality that you otherwise would not have running as a website such as removing security restrictions on cross-domain xhr and requiring permission to display desktop notifications. Of the thousands of lines of code in our extension, only 5% of it is Chrome specific. In theory, this should make porting to other browsers and OS’s easy. Let’s see what it looks like in practice.
A few months ago, we decided to see what our application would look like on the iPad. After just four hours, we had a working prototype of our music player that looked and felt native but more importantly looked and felt exactly like our Chrome extension. This was possible because Chrome and Safari share the same rendering engine – Webkit. Mobile Safari has already implemented many of the HTML5 features we needed such as HTML5 Audio, Web Database and Local Storage and many of the CSS3 features we needed such as Background Gradients, Rounded Corners and Box-Shadows. We were able to re-use most of our Javascript code and since we used a lot of CSS in place of images, we were able to resize elements with ease. But a prototype is just a prototype. Could our web-application actually be a viable shipping product on the iPad?
At this point, the answer is no. Mobile Safari has two quirks that seriously hamper its ability to act as a web platform. The first is it’s lack of fixed positioning. If there was one feature that defined native applications vs. web applications it would be that native applications almost always frame content with a top and bottom layer of buttons or displays. Think about iPhone apps. Almost every one of them (minus games) has a navigation layer on top and a series of buttons on the bottom. To pull this off as a web-application, you need fixed positioning. Unfortunately, Mobile Safari (as well as Android’s browser) do not provide this feature. There are many hacks to emulate it though. Apple has even developed a Javascript/CSS framework to get around this themselves. For our iPad web-app, we used a library called TouchScroll that uses Javascript and CSS animations to emulate scrolling. Overall, the lack of fixed positioning is not a game-ender, as hacks are available, but it certainly adds a layer of complexity to building a client-centric web-app that will make many developers’ lives more difficult. The next quirk may just be a game-ender. Mobile Safari on the iPhone and iPad has an annoying ability to refresh windows once you open up a few. So say you have a few windows open including our music player. The music player is not the active window however. When you click back on it to make it active, it refreshes. I’m not sure why this happens although my guess is it has something to do with memory management. Either way, most apps (especially music or video ones) cannot be taken seriously if the application is constantly refreshing. Sure, in theory applications could maintain their state between refreshes. But in practice, this never works quite as well as we’d like it to. Of course, music stopping and starting constantly will be a game-ender. I am hoping that the iPhone 4 with its 512MB RAM does not have this issue. Android does not seem to suffer from this but Android does not support HTML5 Audio or Video so it certainly has drawbacks of its own.
A couple of weeks ago, Safari 5 was introduced with support for extensions. We spent a few hours reading the documentation to see what it would take to port our extension from Chrome to Safari. Unlike Firefox, where extensions are a bit more than just HTML/CSS/JS and support for HTML5 features are lagging, Safari extensions are theoretically similar to Chrome. Other than different Javascript APIs, the functionality offered is almost exactly the same. We determined that it would not be too difficult to port. Of course, business decisions are not always just about technology, so we are waiting a bit before we do just that. It will however, be very interesting to see if Apple brings extension support to Mobile Safari on iOS.
Coming back to Dion’s original post, it seems like we are closer to write-once, run-anywhere using client-centric web-based applications than we have ever been. With a few modifications, we can basically re-use 80% of our code to deploy on the desktop and mobile. Unfortunately, not all browsers are there yet though. The issues I listed above seem like minor ones that can be fixed easily however. It would not surprise me if in the second half of 2010, Mobile Safari fixes the refreshing issue and Android adds support for HTML5 Audio/Video. As more frameworks are released that deal with the fixed positioning problem, I expect more developers to look seriously at the web as their platform of choice for deploying mobile applications. As for the desktop, Firefox 4 and IE9 should finally add the HTML5 and CSS3 feature support that Chrome and Safari have had for a while now. Of course, not all users will have the latest and greatest browser. Progressive enhancement will be the only way to support those users. But by pushing the envelope of what’s possible, we will hopefully push those users towards upgrading to a modern browser.
For a 60 second run through of ExtensionFM, check out the video below:
David Humphrey and the hit squad of audio gurus have some new amazing demos for us. Perfect for a Friday. This is all through the rich Mozilla Audio API work which will hopefully be pushed into other browsers at some point in the not so distant future.
Charles Cliffe has some awesome WebGL visualizations from Audio. David narrates:
What I like most about these (other than the fact that he’s written the music, js libs, and demo) is that these combine a whole bunch of JavaScript libraries: dsp.js, cubicvr.js and beatdetection.js, and processing.js. Some people will tell you that doing anything complex in a browser is going to be slow; but Charles is masterfully proving that you can do many, many things at once and the browser can keep pace.
Corban and Ricard Marxer have been busy exploring how far we can push audio write, and managed to also produce some amazing demos. The first is by Ricard, and is a graphic equalizer (video is here):
I think that my favourite demo by far this time around is one that I’ve been waiting to see since we first began these experiments. I’ve written in the past that our work could be used to solve many web accessibility problems. A few weeks ago I mentioned on irc that someone should take a shot at building a text to speech engine in JavaScript, now that we have typed arrays. Yury quietly went off and built one based on the flite engine. When you run this, remember that you’re watching a browser speak with no plugins of any kind. This is all done in JavaScript (demo is here, video is here):
In order to do this he had to overcome some interesting problems, for example, how to load large binary voice databases into the page. The straightforward approach of using a JS array was brittle, with JS sometimes running out of stack space trying to initialize the array. After trying various obvious ways, Yury decided to use the web to his advantage, and pushed the binary data into a PNG, then loaded it into a canvas, where getImageData allows him to access the bytes very quickly, using another typed array. The browser takes care of downloading and re-inflating the data automatically. Here’s what the database looks like:
My favourite line is:
What began as a series of experiments by a small group of strangers, has now turned into something much larger.
What an awesome community you guys have… and we are all benefitting. Thank you.
Makinde Adeagbo is describing Facebook's Javascript performance optimisation efforts at JSConf. By 2009, it became clear something had to be done, as the trend was towards longer and longer page loads. Back in 2006, the objective had been super fast page loading: "If any page loads in under 100ms, it takes way too long". By 2008, given all the new interactive features on the page, the 100 millisecond target had expanded to 1 second, and by mid-2009, pages were taking 5 seconds to load. Even HipHop, the PHP compiler, was smaller than the Javascript code base. And when Steve Souders called out Facebook in his book numerous times, it only adding to the team's sense of fun and joy ;).
In June 2009, the team quickly chose a target by year's end by simply checking the load time without Javascript: 2.5 seconds. Thus, dropping load time from 5 seconds to 2.5 seconds was the goal for end of 2009.
The first initiative was to include the Javascript at the bottom of the page. Great, it's faster, but at what cost? A big one: Users try to click on controls, and nothing happens. Back to the drawing board, and the team refined the setup so that the actionable stuff was initialised on the top of the page. But how to minimise all this code at the top of the page? Here, the team exploited the observation that most controls work the same way:
* User clicks
* Sends ASync request
* Insert/Replace some content
So the team set up elements like this:
<a href="/ring.php rel="dialog">...</a>
... And then hijacked them with a standard listener routine, one that would work for most of the controls. (Most, not all; 80/20 principle is in effect here.) This way, they could have one small listener routine to handle most of the controls on the page. Once the user clicks, the server gets called and outputs a new snippet of Javascript:
$response = new AsyncResponse(); $response->setContent("#elem_id", $new_content); $response->appendContent("#content .stories", $new_story); $response->send();
And they are using a convention: "ajaxify=1" on an element indicates it's ... Ajaxified.
At this point, the team had now Ajaxified a bunch of features, but people were still skeptical about more complicated features. For example, would setting status be too hard with the same techniques. So after some research, Makinde came back with an epiphany: the humble form. Whereas the previous async requests were effectively information-less - just a simple directive and maybe an ID - the requests would now include content too. And of course, most of these things look nothing like forms due to styling. But underneath, they're still forms, e.g. the entire comments block is a single form.
Nowadays, most of Facebook runs without complete page refreshes, by dynamically flipping the content and the fragment ID. (What Facebook calls page transitions.)
Ongoing, Makinde says performance optimisation requires ongoing vigilance. Every engineer has their special case, but in the scheme of things, it's better to say no to new features unless they can be strongly justified. For example, we can live with user submitting a form that hasn't yet been hijacked; a complete page refresh is fine on occasion. We don't like it, but we don't want to make it a special case just for the sake of it.
The Gantt charts tell a great tale: users now see content much earlier, and it's interactive. So how did they fare with that 2.5 second goal for year's end? Achieved by December 23. And Makinde wants people to know Facebook is hiring as they have more Javascript to write...and delete.
Every Time Zone is a great new Web app by Thomas Fuchs and Amy Hoy that shows how the Web platform is ready to create fantastic visual experiences. In the spirit of "do one thing, but do it well" the application shows you what time it is in various parts of the world. Now when I call folks back in blighty, or bug Rob Castaneda while in Australia, I can get a quick view of the time.
If you take a peak on the iPad and add it to your home screen, you will see that it runs very nicely offline indeed. The rich UI effects are top quality, and make use of CSS gradients and the like to power a desktop class utility.
I always enjoy looking at their code too. Nice and clean:
We have seen a lot of playgrounds out there with JS Fiddle being the most recent. The latest is CSS Desk. It is an incredibly simple playground where you put in HTML and CSS, and see a live preview. That is it.
However, Pixel Matrix Design... the folks behind it... show that they design very clean and beautiful web apps.
It looks and feels SproutCore/Cappuccino-esque, but it is actually simple markup with sprinkling of jQuery. Nice animations and drop over effects. Click on the toggle buttons. Click on the arrows for drop downs and panel show/hide. Very nice.
Watch the video above by Volodya Kolesnikov, the developer behind the Mail.app prototype that uses the UKI framework.
ukijs matured and I put it to the test. The idea for the test was to replicate Mac Mail.app core experience in the browser. With as much attention to details as possible and acceptable performance. The whole thing took me about 15 hours and 65kb of gziped code.
<divclass="ajTourDef"id="defaultTour"style="display:none"title="http://amberjack2.org/"> <divtitle="http://amberjack2.org/"> <divtitle="{id:'logo',padding:15,trbl:'brr'}"> <pclass="title">Amberjack allows easy creation of On-Page Tutorials</p> <br />
Navigate with bubble arrows, keyboard right/left or mouse click left/right </div> <divtitle="{id:'tryDemo',padding:0,trbl:'brl'}"> <pclass="title">Text bubbles explain page elements to your visitors.</p> <br />Use your <button>HTML</button> and <spanstyle="color:#ff0;background:#000">CSS</span> knowledge to customize the bubbles as you like! </div> <divtitle="{id:'tryDemo',padding:0,trbl:'brl'}">
Embed a Youtube Tutorial for example <br /><br /> <objecttype="application/x-shockwave-flash"style="width:330px; height:205px"data="http://www.youtube.com/v/xlroni8b6GU"><paramname="movie"value="http://www.youtube.com/v/xlroni8b6GU" /></object> </div> <divtitle="{id:'markerBasics',padding:-2,trbl:'blr'}"> <pclass="title">Jump DOM elements with their unique DOM ID</p> </div> <divtitle="{id:'markerPositioning',padding:-2,trbl:'ltb'}"> <pclass="title">Place multiple tours ...</p> </div> <divtitle="{id:'markerColor',padding:-2,trbl:'ltb'}"> <pclass="title">... on your page</p> </div> <divtitle="{id:'techcrunch',padding:5,trbl:'tlr'}">
Amberjack was covered on Techcrunch... </div> <divtitle="{id:'ajaxian',padding:5,trbl:'tlr'}">
... Ajaxian ... </div> <divtitle="{id:'digg',padding:5,trbl:'tlr'}">
... and even got Digged </div> <divtitle="{id:'delicious',padding:5,trbl:'trl'}">
Please add Amberjack to your bookmarks </div> <divtitle="{id:'uservoice-feedback-tab',padding:0,trbl:'rmb',position:'fixed'}"> <pclass="title">That's it, please leave Feedback</p> <br />
Close tutorial by clicking the cross (X) above or hit ESC on your keyboard. </div> </div> </div>
You can check out the source and more tutorials and documentation is coming.
Ben and I are often touting the fact that we are about to see amazing Web applications as browsers support more and more of "HTML5" and the engines under the hood have become world class.
The GWT team has developed something that truly shows off this engine in their port of Quake2 to GWT. Seeing the game run with amazing fps in a browser, with online multiplayer network support via WebSockets, is something to truly behold.
Stefan Haustein, Ray Cromwell and Joel Webber were at a GWT summit and happened upon Jake2 the Java port of Quake2 that runs on top of JOGL for OpenGL. Ray could see that Joel had a glint in his eye, and the porting began!
To get the rich 3D graphics they went for WebGL using a GWT wrapper library that Stefan had built. Jake2's heavy use of java.nio.Buffers were handled by implementing JRE classes that mapped those to WebGL typed arrays. File system code that used RandomAccessFile was emulated by using HTML5 LocalStorage. The OpenAL audio system was implemented using HTML5 audio.
The various team members got varying fps depending on their machines. A Linux notebook managed 60fps, a Mac Pro got 45fps, and a Macbook Pro got 25fps. WebKit was able to perform the best of the browsers right now because it doesn't have the multi-process per tab tax that means a lot of OpenGL buffer copying.
I think this is a big deal in a couple of ways:
You can build fantastic games that use rich 3D engines and fast network access right on the Web. How do you "join a game"? You just share a URL! Tweet away! Now you can have these rich games just a URL a way. No install required.
Game engines push a platform. Although we are talking about building an older game engine (doubt that the latest and greatest engines would run like this!) this is PLENTY for building rich effects and applications. Take some of this richness and look at something like Google Docs. How much smoother and nicer can it be? We can create beautiful applications with effects like butter right now.
Credit
Ray Cromwell (who posted on this here) contributed audio/mouse/keyboard and localstorage for prefs, plus some minor GL shader debugging. Joel Webber and Stefan Haustein did the other 80% (Joel did the majority of the original code surgery to get the multiplayer game running under GWT minus graphics, Stefan did the heavy lifting on WebGL and singlehandedly ported the single-player game logic)
People still claim that "the browser is slow" and "JavaScript is slow". Not in 2010 my friends!
Who needs Native Client now when you can build apps and games like this! Well, to be fair, there is a niche of folks who want to use C libraries and frankly prefer that world... so NaCl is for them.
For the rest of us though? Our world keeps getting better, almost daily!
David Friedman calls is "silly", but he has created something simple and fun in his new form of volume control that moves from whisper to shout as you change the volume.
The code (which works on Firefox and Opera right now) uses different tracks for each level:
function timeUpdate(){ //get the duration of the player
dur = audio_player.duration;
time = audio_player.currentTime;
fraction = time/dur;
percent = (fraction*100);
wrapper = document.getElementById("duration_background");
new_width = wrapper.offsetWidth*fraction;
document.getElementById("duration_bar").style.width=new_width+"px"; }
This has been said because Ext is a very high level language for Web development. It is almost DSL like versus offering low level functions that you can sprinkle in your code. If you buy into the Ext way then you can build UIs in a very simple manner through JSON. This is all perfectly setup for a design tool, and with Ext Designer, they have it.
Feature Highlights
Fast prototyping of the look and feel of an application. Ext Designer enables developers and designers to assemble robust layouts by dragging and dropping their components together.
Fine-tuning without hand coding. Easily change the layout of a specific component, or use the inspector to fine-tune detailed properties.
User client-side data stores. Ext Designer includes several new ways to incorporate and configure data stores, which can then be bound to components like DataGrids and Trees. Whether your data is in JSON, XML, a simple JavaScript array, or an Ext Direct connection, Ext Designer can help you build a data store for it faster than ever.
Compatible with other development tools. Once the application design is finished, code can be exported on a component-by-component basis, as either JSON or JavaScript classes. Ext Designer creates tight JavaScript that can be imported into any popular IDE.
Interestingly, the took is a desktop application versus a Web app like Bespin, CodeMirror, or Ares. Some will be happy with this, as it gets the benefits of a desktop tool.... but I am a fan of where Ares in particular innovates (it lets you easily integrate your desktop files with the cloud.... via a Java applet).
If you already develop on Ext, it is a no brainer to give the designer a good walk through. At $219, if it saves you a couple of hours of work it is probably worth it :) The fact that Ext offers layout, rich components, and service integration, means that you can get an app up and running in short order.
Harmony is a new drawing tool, a HTML5/Canvas experiment with great potential. It provides some unique brush styles, and can produce some great-looking charcoal pencil style sketches, among other things. Better to try it out than explain it in words.
Creator Mr. Doob (Richard Cabello) explains how he used Canvas to make it darker the more you draw over it:
The whole thing is quite modular so I can keep adding more brush styles whenever I get inspired. During the process I found out that, for some reason (apparently lack of hardware acceleration), Firefox and Opera do not support context.globalCompositeOperation = 'darker'. This was on the HTML5 spec before but got removed. Just so you know what I'm talking about, this is like the "multiply" blending in Photoshop. Webkit does support it tho. I hope they put it back on the specs and all browsers support it.
You can also save images using data URI encoding.
As it works on webkit, he made sure it worked on the mobile Android and iPhone browsers. No multi-touch as yet, but the touch UI still makes a nice input mechanism.
Bastian Allgeier has developed a beautiful, native looking web application called ZooTool.
Zootool is a visual bookmark tool for images, videos, documents and links. It is completely based on Mootools, even though it looks more like a Cappuccino app!
eBooks have gone mainstream, and right now the open ePub format is getting a lot of attention, being the iPad's book format of choice. Often overlooked in gadget-centric media is the fact that ePub is based on web standards, and therefore amenable to being rendered in the browser, sans plugins. Pure Javascript ePub readers are starting to crop up, and Keith Fahlgren has written about several of them:
Just in the last few days, details emerged of two new JavaScript ePub readers, rePublish from Blaine Cook (@blaine) and JSEpub (screenshot) from August Lilleaas (@augustl). These two new readers join @liza’s epubjs, which will be a year old on Tuesday. An improved version of epubjs powers the ePub Zen Garden, which helps “dispel the myth that digital books can’t also be crafted works of visual design.”
All are open source, and as Keith notes in the comments, there's also the commercial BookGlutton project. BookGlutton (which we covered earlier) shows the promise of browser-based eBooks: it lets you embed books as lightbox-powered widgets, and supports annotation.
EPUB consists of three specifications:
Open Publication Structure (OPS) 2.0, contains the formatting of its content.[5]
Open Packaging Format (OPF) 2.0, describes the structure of the .epub file in XML.[6]
OEBPS Container Format (OCF) 1.0, collects all files as a ZIP archive.[7]
Basically, EPUB internally uses XHTML or DTBook (an XML standard provided by the DAISY Consortium) to represent the text and structure of the content document, and a subset of CSS to provide layout and formatting. XML is used to create the document manifest, table of contents, and EPUB metadata. Finally, the files are bundled in a zip file as a packaging format.
In the case of unzipping, Keith points out the inflate library has been around since 1999. One can imagine other applications for zip too; for example, it's often used as a format for bundling code (Java JARs, Python Eggs, Firefox and Chrome extensions), so reliable unzipping makes it possible to build browser-based IDEs and exploration tools against such archives.
Román Cortés is having a lot of fun with CSS tricks these days. He just built an example rolling CSS coke can that uses background-attachment, background-position, and a few other tricks to get the effect. No fancy CSS3 needed here!
