A Googler and a Facebooker were in a pub discussing the complexities of building out a rich modern Web application. There are a ton of dependencies, and you need to be proficient in multiple languages and tools (JavaScript, HTML, CSS, SQL/NoSQL, backend languages, build tools, etc).
Well, they may not have been in a pub.... but a deadly duo did get together to try to solve this problem.
All of us on the Asana team have deep backgrounds writing rich web applications at companies like Google and Facebook. We've been continually frustrated by how long it takes to write software, and by a nagging feeling that in some deep sense we've been writing the same code over and over. Even when using the latest and greatest frameworks and disciplines, writing fast, highly interacting web applications involves a lot of accidental complexity:
First you need server code to figure out what data the browser needs. Hopefully you have an ORM layer, but you still need to carefully structure your code to minimize your backend dispatches, and you need to carefully keep that in sync with your front-end code lest you don't fetch enough data or hurt performance by fetching too much. If it's a Web 2.0-style app, you re-implement a ton of that server-side code in JavaScript, once for creating the page and then again as controller code for keeping it up to date and consistent. And when the user changes something, you bottle that up -- typically in a custom over-the-wire format -- and send it as an XHR to the server. The server has to de-serialize it into different data structures in a different language, notify the persistence store, figure out what other clients care about the change, and send them a notification over your Comet pipe, which is handled by yet more JavaScript controller code. Offline support? More code.
This is not a one-off task: it's rote work that adds complexity to every feature that you build in every application. By the time that you are done with all this, the important and novel parts of your application are only around 10% of your code. We wondered whether we could build a programming system in which we just wrote that 10% -- the essential complexity -- and a compiler handled the other 90%.
And this lead them to their solution:
Inspired by incremental computing, we're building Lunascript as a simple way to write modern web applications. Lunascript has a syntax and easy of use reminiscent of JavaScript, but a powerful pure-functional lazily-evaluated semantics historically confined to academic languages.
A Lunascript application specifes a data model and a function from the model to the view or user interface, annotated with handler functions from user inputs to model mutations. From this the Lunascript compiler produces a functioning Web2.0 application -- the client-side JavaScript, the server-side SQL, and everything in between -- complete with real-time bidirectional data synchronization. There's no need to write separate code to help the server figure out which values need to be sent to the client: the server can do this by simulating the UI. Because a LunaScript application only specifies how the UI should look given the current data (rather than how the UI should be updated as changes happen) it's impossible to write a UI that loads correctly but does not stay correct as changes are made.
"What does it look like?" I hear you cry. Time for a hello world, and since Comet is in the mix, that means a chat server:
You will notice the protobufferness at the top, and the fn of less characters, and E4X-like.
To learn more, let's listen in to Dustin as he gives us a walk through the world of LunaScript:
I talked to the guys and asked a few questions which they answered..... what questions do you have for them though? Are you excited about what a higher level abstraction could give you? Or do you like to be close to the metal?
Christophe Eblé has kindly written a guest post on Swell JS and his drag and drop manager that works with your desktop. Here he tells us more:
At Swell we were about to create a Drag & Drop Manager just like in other Javascript libraries such as Jquery, YUI, Mootools, Scriptaculous, but we were not really satisfied with this decision.
What motivated us to adopt another strategy is that we didn't want to provide yet another simulated solution but instead a drag & drop library that would use the real power of the web browser.
We've been faced to a lot of problems and we are still struggling with API differences. The Pros and cons of using system drag&drop over simulated solutions:
Pros
Accuracy and performance, mouse move tricks to position an element under the pointer and detect target collision are things of the past, system DD is wicked fast!
system DD can be anything you like (simple images or complex dom nodes) and can interact within your browser window, the chrome (search field, address bar...) or tabs (if the browser allows it, FF 3.5 does it right) and even your OS.
system DD through the dataTransfer object can carry powerful meta information that are not necessary the dragged object itself, this can be arbitrary text (JSON data for ex.), urls and for the latest browsers complex data types see this
system DD has true DOM Events
Cons
Browser differences in this subject are a real pain, I couldn't list all the oddities here :)
The drag & drop implementation in Swell is still at an early stage, and not officially released but here's some details on what we're working on.
Provide a single way to create native drag & drop objects in every possible browser
Provide setDragImage method on browsers that don't support it yet
Provide a DD Manager that acts as a container for all drag and drop events on which you can place your handlers, cancel events, or quickly batch all the DD objects on the page. (useful when you deal with dozens of DD Objects)
Provide a way to associate complex metadata with each DD objects
Provide a way to easily create visual feedback for your DD Objects
There's a TRAC available on the project with a roadmap and release dates, a blog, and even a SVN repo that you can check out. Be careful, as I said above the library is still in heavy development and is missing docs! We are looking for any kind of help and just hope to receive massive feedback ;).
See some examples in action:
In the video, we are providing a simple yet powerful application to import a RSS feed in your webpage. The classical way is to type in the feed URL and then getting redirected to it, which commonly takes 3 to 5 steps. With this implementation you just have to drop the RSS icon onto an appropriate target and that’s all folks!
We use YQL and JSONP to transform RSS into JSON and of course a Swell Element to dynamically attach the behavior to the webpage:
Sergey Ilinsky, formerly of Ajax pioneer Backbase, has created a new Ajax toolkit called Ample SDK. The best way to describe Ample SDK is that it's a browser-within-a-browser, but not in the visual sense. Rather, it aims to implement in JavaScript the stack of rendering technologies typically delegated to the browser. For example, Ample currently lets you use Mozilla's XUL to create cross-browser applications.
Here's an example of how you could use Ample SDK to render a XUL interface in the browser:
<body>
<!-- other HTML code -->
<script type="text/javascript">ample.open()</script>
<xul:menulist xmlns:xul="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul">
<xul:menupopup id="items-popup">
<xul:menuitem label="Item 0" value="0"/>
<xul:menuitem label="Item 1" value="1"/>
<xul:menuitem label="Item 2" value="2"/>
</xul:menupopup>
</xul:menulist>
<script type="text/javascript">ample.close()</script>
<!-- other HTML code -->
</body>
(Sorry for the old-school, ugly <pre> above--our code formatter chokes on XML namespaces, used in the XUL elements above.)
The Ample SDK compatibility table, shown above, gives you a sense of the scope of the project: to implement technologies like XUL, SVG, HTML 5, XHTML, and so forth, all in JavaScript--and render them across modern and legacy browsers like IE 5.5.
When Brendan Eich first hinted several years ago that the browsers would get fast, it spurred a lot of folks to wondering whether the future of feature innovation would be primarily in the JavaScript library space, and the browsers would focus on being JavaScript run-times and graphics abstract layers.
Ample SDK raises the question anew. Should we push the rendering layer to JavaScript libraries? What do you think?
Sofa has been working on a new look, and Aristo has been released to the world via PSD. It is very cool that the look and feel has been open sourced.
All the controls have been updated to use the new style, and we’ve added a few new controls as well, like checkboxes, radio buttons, and segmented controls.
Theming
We didn’t just build one new look for Cappuccino, though. Instead, we built an entirely new theme engine which will let anyone create a theme for Cappuccino. All the properties of the standard controls are fully themeable in all the different “theme states” (like “pushed”, “disabled”, “normal”, etc.). In the coming months, we’ll be sharing new themes with the community, and building a site for users to post their own themes for others to download and use.
Interface Builder
Those of you who come from a Cocoa background are used to using Apple’s Interface Builder tool to layout your Cocoa applications. Thanks to a technology in 0.7 called nib2cib, you can use Interface Builder to layout Cappuccino applications too. All of the classes supported in Cappuccino 0.7 that have Cocoa analogues are included. So, you can drag checkboxes, sliders, text fields, buttons, and lots more. You can even instantiate custom top level objects, or custom view subclasses, which will be converted to the right class in Cappuccino by the nib2cib tool. Once you’re done laying out the UI of your application with Interace Builder, you can use it’s target-action and outlet technology to build up its logic as well.
It is also cool that Narwhal an early implementation of the ServerJS work has made it in there. Great work guys!
Talks about JavaScript are not a rare occasion any more, everybody has something to say about this wonderfully versatile language. Douglas Crockford taught us a lot about the language itself, John Resig and Peter Paul Koch taught us how browsers deal (and fail dealing) with it and there are dozens of screencasts of how to use this or that language.
What most of these talks and videos have in common is application of JavaScript and how to deal with issues that will occur. Information about designing and architecting large JavaScript solutions on the other hand are rare. Satyen Desai now broke the silence and gave a great talk on the architecture and design decisions of the the third generation of the Yahoo User Interface library:
With the release of IT Mill Toolkit 5.3.0, the server-side RIA framework is now ready for production. I announced the initial release of 5.0 back in December of 2007. Since that time, IT Mill 5 has gone through several revisions and the release of GWT 1.5 (which means you can use Java 5 now on both the client and the server). As a reminder, server-side RIA frameworks let you write your app completely in the server and uses a client-side Ajax engine to render the interface. The nice wrinkle with IT Mill is that both the server side and the client side are written in Java, so if you want to add a component, you don't have to break out the JavaScript (see the extensive and high quality reference manual for details on how to develop your own custom components in GWT). If you're a Java shop, that's got to be a good thing.
As it turns out, the Cappuccino team has been busy hacking away on some very impressive stuff. Today at the Future of Web Apps Miami, they announced Aristo and Atlas.
Aristo
Cappuccino worked with the popular design firm Sofa to create Aristo, a new open-source look-and-feel that will be freely available, including the source PSD files complete with all layers intact. Anyone may use the look in their own projects. I didn't get a close look at the theme, but it seemed quite nice.
Atlas
Atlas continues Cappuccino's efforts to essentially port Apple's Cocoa stack to the Web. Atlas itself looks like a mash-up of Flex Builder and Interface Builder, all running in a browser, but continues to hew very closely to Interface Builder's approach. For example, just as with Interface Builder, you can bind properties on components to other components or to code through visual drag-and-drop operations (complete with the same blue connector line).
What's more, Atlas introduces the equivalent of Interface Builder's "nib" file (though I forget what their version is called).
In a live demo, Francisco Tolmasky built an RSS reader using an entirely visual approach. Francisco made it clear that he doesn't think code will go away, but instead, he feels the amount of code required to create a web application will dramatically decrease.
They also released a screencast:
Multi-platform Support, including iPhone JavaScript Wrappers
Atlas also supports targeting multiple platforms, letting you create different interface files that bind to the same back-end code. They've even bridged native APIs and exposed them to the JavaScript environment and provide a PhoneGap-like way to run such applications as native iPhone applications. Wow.
It's hard to overstate how impressive the Atlas demo was; the release should be coming at some point in summer 2009.
The good folks at Xopus (the really cool browser-based XML editor) have been working largely in silence on a new version of their application. They've recently posted a presentation that talks about the framework behind this new version of Xopus.
In the linked PDF, the framework is revealed to be an "object-oriented 'extended subset' of JavaScript", much like Objective-J, but while Objective-J brings a syntax very similar to an existing language (Objective-C), the Xopus framework brings its own syntax to the game:
function favorite (){returnnew Food("banana") ; },
Static, function kind ()return"chimp" ; }
);
Among the features of the Xopus framework are:
Hierarchically structuring programs into packages
Some forms of program veri?cation
Making dependencies explicit
Dependency resolution
Consistent ?le-system layout
Server side compilation to ?attened form
Serving the client efficient and possibly obfuscated code
Preserved stack trace for easier client-side debugging
Inheritance
Annotations, including visibility (public, private, protected), static, continuation, test, deprecated, API--they are used for documentation and compilation and stick around at run-time for reflective access
The server-side compiler is presently based on SpiderMonkey; the compiler is entirely written in JavaScript and includes "extensions for profiling, coverage, and dependency visualization."
Unit Tests
Unit testing is baked into the class format, as in:
New language addition to Objective-J: The @accessors keyword to eliminate the boiler plate code of writing setters and getters. Read more about it here.
Support for more classes like CPTimer.
Performance improvements all around, but especially in Objective-J thanks to a major rewrite of the parser.
Improved debugging tools and a new debugging “mode”.
Over at the YUI blog Satyen Desai just announced the release of YUI3 PR2. The main changes are that the widgets from YUI2 are now all getting ported over to the new, leaner YUI3 architecture and that the team made it much easier to write your own YUI widgets and plugins extending the existing ones.
Mozilla is big on tinkering--making things your own. They recently released a marketplace for their community to upload its own shirt designs. But because they based it on the Ajax-heavy Zazzle platform, consumers can customize the shirts in a variety of ways. We thought this made for a good opportunity to take a closer look at Zazzle.
Easy Zooming
In the overview page showing different shirt designs, simply mousing over a shirt lets you zoom in on it in a smooth, easy motion.
Clicking into a shirt gives more Ajax goodness, like an alternate zooming interface showing the full-view and the detail simultaneously:
And finally, it has a shirt manipulation interface, though most of the design manipulation occurs on the server, it is sent back without a page reload, as we would expect:
Zazzle is built on the UIZE (pronounced "you-eyes") Ajax framework which we covered briefly in 2006 and have yet to revisit. We're not sure if UIZE is used by anyone but Zazzle, but it's got a pretty nice website, docs, widgets, basic effects, an event system, and an inheritance system. We'll have to give it a closer look!
Two years ago we covered JotForm, a Ajax-based web form builder. The folks behind it have been busy in the intervening time and have now released a more general tool: Memberkit.
Three years in development, Memberkit lets you
use the drag and drop tools to create listing pages, display pages or search forms. You can create multiple forms and create relationships between them... without writing any code.
While at present Memberkit targets a particular niche, it has ambitious plans.
We have released this first version specifically for membership and
social sites. Our plan is to first make it perfect for these specific
taks before turning it into a complete web framework.
Changes to the app remain persistent because on the server, the xml object has changed.
Of course, different users gets different versions of the same widget, because they have their own xml runtimes.
A widget can contain any amount of javascript to execute clientside, and also include any css to style it, like this calendar application.
A widget can also split itself into subsets, this example opens a new widget as a dialog, which is really a part of the same widget, and saves data back to itself from it.
XML objects can access each other on the server within the boundaries of the same account.
It means that widgets can access other widgets and send/recieve data from them.
The xml for this app in this example, which is behaving much like a webservice, is here.
File system access
Widgetplus allows complete filesystem access for a widget, but only within the user account.
By default, widgets always access the demo account, but when logged in, they access the users home account, it’s files and folders, and other settings.
This is crossdomain by default, like this example on Blogger, which accesses all files on the demo account.
These two parts, together, define what widgetplus is.
It’s a serverside xml runtime for widgets, which can access and manipulate files and folders that belong to the free user account.
Users can always sign up / log in to any widget (by default), the developer wont have to bother about that.
Widgetplus clones all widgets as soon as someone signs up, and also automatically creates a webspace directory for the new user with the default folders in place.
Otherwise, whenever someone logs in, it simply serves the requested xml object(s) associated with that user.
It’s important to note that widgetplus isn’t a BOX.net service, the webspace is for widgets - and whatever they do with it – it’s not for uploading your entire music collection. We will move to a server network in about two months, after which we plan to offer unlimited file storage, but for now, we need to keep a tight leach.
ZK 3.5, the latest version of the server-side Ajax framework, is out with a raft of new features. Three of those features really stand out for me:
Comet server push
Customization of look and feel
Performance monitoring
Server push via polling has been available in ZK for a while, and Comet in the ZK "Enterprise Edition," but now it is available to everyone. And it is pretty easy to use: "The implementation of server push is transparent to developers. ZK chooses which implementation to use according to the edition of ZK automatically, but it is configurable."
Customization of look and feel has gotten much easier. ZK has followed the example of a number of other frameworks in styling its widgets with predictably named CSS styles. Changing the look and feel of an application is now as easy as changing the ZK widget style sheet. Styles can further be overridden on a widget instance-by-instance basis.
Performance monitoring is perhaps the most exciting new feature. Client-side tools such as YSlow can guide optimization efforts and give you point in time performance snapshots. But critical applications need to be monitored and tracked end to end over their lifespan. With ZK 3.5, you now have the plumbing to instrument your application to capture five data points for each request:
T1, the time browser sends a request to server
T2, the time server receives a request
T3, the time server sends a request to browser
T4, the time browser receives a request from server
T5, the time the browser finishes processing a request
One of the reasons Dion is such an effective editor here at Ajaxian is his sense for filtering all of the available news from the Ajax community down to about three stories a day. Truth-be-told, with all the submissions we get and what we find on our own, we could easily post 10 stories a day. But in today's saturated environment, we find three stories is about the right number to keep from overwhelming our readership with too much noise. We hope you agree, and we're interested in hearing if you feel otherwise.
A consequence of this arbitrary filtering is that some of the lesser known frameworks and libraries simply don't get covered. We never try to be king-makers, nor do we have that kind of clout--we simply can't cover everything.
Creating a lot of HTML using DOM methods can be a real pain. This is what students of the Juku training course that I held two weeks ago found out quite quickly and complained about the verbosity of it all. I listened to their concerns and came up with a framework for JavaScript applications called ViewsHandler.
One of the tasks I had given the class is to create a thumbnail show with image information from a link pointing to Flickr using the JSON API. The following is a solution using and showing the options of ViewsHandler:
A flickr show created with ViewsHandler
ViewsHandler is not another JavaScript templating solution but works on the assumption that in most cases you'll have to create a lot of HTML initially but you'll only have to change the content of some elements dynamically as new information gets loaded or users interact with the app. So instead of creating a lot of HTML over and over again all I wanted to provide is a way to create all the needed HTML upfront and then have easy access to the parts of the HTML that need updating.
The first thing you'll need to do to define your application is to create an object with the different views and pointers to the methods that populate the views:
ViewsHandler then creates DIV elements for each of these views and hides them for you. In your methods you create all the HTML the different views need to have and apply it with an add() method. You then define the parts of the HTML that should change later on as fields using the define() method and you can use the set() method to change the content of these fields and the view() method to change between views.
The benefit is that for setting the data you don't need to access the DOM any longer or use innerHTML or nodeValue. ViewsHandler created a pointer to the element all of this is cached. The set() method also allows you to either add a new node as the value or a string. In the latter case it'll create a text node for you.
One convenience method is linkto() which creates links pointing to the different views for you. None of this is rocket science, but it helped the class to create large applications with complex views without losing track of what they are doing. Maybe it can help you, too.
