Enter Dav Glass of the YUI team. He’s been turning the concept of progressive enhancement around in his head and as a JS lover and backend code “endurer” he set out to solve this issue once and for all in a pure JavaScript way.

Here’s the issue: you cannot assume JS to be available on the client side as your visitors might be on a slow mobile connection or are paranoid enough to turn off JS or actually have it turned off for them by company policy or many other cases.

So in order to use pure JavaScript to render a solution that works for everybody the natural thing was to move the JS solution to the server side and re-use it on the client when it is possible.

If you say server-side JavaScript you end up quickly talking about Node.js and so did Dav.

Check out the following demo:

All of what you see is driven by JavaScript – there is no server-side PHP fallback. Yet if you turn off JavaScript in your browser, you get the same experience.

The reason is that Dav uses Express.js and Node to render the HTML of the application server-side with JavaScript and YUI3. The code is available on GitHub and a video of him giving a talk about this is available soon on the YUI blog.

This is an exciting concept as it means that we can build progressively enhancing solutions using any widget library if we just think of the server-side case, too. This can mean a lot less code and easier maintenance as the only skill needed to build bullet-proof solutions is JavaScript.

The missing piece to the puzzle though is a fully fledged DOM renderer on the server side. DOMJS works but a C++ version would be much better in terms of performance and have all the features of DOM-2.

In his own words:

Software development is hampered by a specific set of design mistakes that were made in the first programming languages and repeated in everything that has been done since. And, somewhat miraculously, JavaScript is going to make it right, radically transforming the way we write applications. Again. In the Loop of History, it has all happened before, but it has never happened like this. This is why you should care about the emergence of server-side JavaScript and the excitement around projects like Node.js — not because they’re at the sharp end of a trend, but because they’re paving the road toward the next big revolution in software.

You can sign up for your free ticket on Meetup and there are still a few places left. As with all the BayJax talks the video will be available later on, too.

See addmap.js in action below - all the content in the green box is generated from the paragraph of text above it. You can try it out for yourself by clicking the screenshot.

Using addmap.js is easy - sign up for a Google Maps Key and provide it as a configuration parameter. Then call the analyse function with the ID of the element to analyse as the parameter:

The script uses YQL and Yahoo PlaceMaker under the hood, for more info and updates on this, check the blog.

This loads the document any link with a class of "ajaxtrigger" points to and updates the content of the element with the ID "target". If the link is a third party link on another domain it fails though - and silently at that. Normally you'd work around that with a server-side proxy, but you can actually do without it.

YQL is a hosted web service that can scrape HTML for you. It also runs the HTML through HTML Tidy and caches it for you. For example to load http://bbc.co.uk you'd use the following statement:

As a URL this turns into:

You could request JSON-P by setting the output format to json and define a callback, but that would give the HTML back as a massive object which is not nice. YQL also offers JSON-P-X as an alternative which is a JSON object with a callback and the HTML as a string inside a simple Array. See it by clicking the following URL:

http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%27http%3A%2F%2Fbbc.co.uk%27%0A&format=xml&diagnostics=false&callback=foo

Now, using jQuery's getJSON() we can load this even without a named callback function. That way we can use one method for content that is third party and simply use load() for the other:

You can see the demo in action here, more details are available on my blog and the source is available on GitHub.

Douglas Crockford is Yahoo!'s JavaScript architect and a member of the committee designing future versions of the world's most popular programming language. In the first three months of 2010, Douglas will be delivering his acclaimed series of lectures on the history of JavaScript, its features, and its use.

1. Monday, January 25: History of JavaScript (RSVP)
2. Friday, February 5: Survey of the JavaScript Language (RSVP)
3. Wednesday, February 17: Functions and Inheritance in JavaScript (RSVP)
4. Wednesday, March 3: JavaScript and the DOM (RSVP)
5. Wednesday, March 31: Style and Performance in JavaScript (RSVP)

Lectures will be held in URLs Café, Building C, on Yahoo's main campus in Sunnyvale, CA. Doors open at 5:30 p.m.; lectures begin at 6 p.m. Pizza and refreshments will be served. Attendance is free, but seating is limited; RSVP is required for admittance.

I can't wait!

Building a search mashup with YQL using Google, Yahoo and Bing - live :) from Christian Heilmann on Vimeo.

Give it a lick of paint and you have a pretty sweet little tool:

All the source code is available on GitHub.

The YUI team has put out YUI 3.0:

We’re pleased to announce today the general-availability release of YUI 3.0.0. YUI 3’s core infrastructure (YUI, Node and Event) and its utility suite (including Animation, IO, Drag & Drop and more) are all considered production-ready with today’s release.

This is a ground-up redesign of YUI:

1. Selector-driven: YUI 3 is built around one of the lightest, fastest selector engines available, bringing the expressive power of the CSS selector specification into actions that target DOM nodes.
2. Syntactically terse: Without polluting the global namespace, YUI 3 supports a more terse coding style in which more can be accomplished with less code.
3. Self-completing: YUI 3’s light (6.2KB gzipped) seed file can serve as the starting point for any implementation. As long as this seed file is present on the page, you can load any module in the library on the fly. And all modules brought into the page via the built-in loader are done so via combo-handled, non-blocking HTTP requests. This makes loading the library safe, easy and fast.
4. Sandboxed: YUI modules are bound to YUI instances when you use() them; this protects you against changes that might happen later in the page’s lifecycle. (In other words, if someone blows away a module you’re using after you’ve created your YUI instance, your code won’t be affected.)

It's especially nice to see the new terse YUI namespacing, so you can just type YUI() instead of the older longer syntax.

The cool thing about YUI (and this release) is that it is literally driving the Yahoo! Home Page. That's pretty awesome of Yahoo! to release this code and make it generally available to the wider community. Congrats to the whole YUI team on the new release.

See the original announcement blog post on getting started with YUI 3.0 in 3 easy steps.

Some of the highlights of the new release are four new components - the Storage Utility, SWFStore Utility, SWF Utility, and the ProgressBar Control.

The Storage Utility is a wrapper for local storage supporting Google Gears, Flash or HTML5 and was submitted by Matt Snider who works for Mint.com.

The SWFStore utility, written by Alaric Cole provides a predictable API for interaction of Flash and JavaScript whilst Allen Rabinovic's SWF Utility is a more granular way of embedding Flash in web sites than SWFobject.

The ProgressBar control by Daniel Barreiro aka Satyam is another example of just how detailed YUI controls are when it comes to exposing custom events. This allows you to either use it out of the box or extend it by overwriting only what you need rather than messing around with the main code of the control.

Other noteworthy changes in YUI with the 2.8.0 release are an extension to the event handling component to support event delegation without having to write it in detail, the connection manager supporting crossdomain requests and getting cut up into smaller on-demand components, and the Carousel control getting a "gallery style".

Check out the more in-depth overview on the YUI blog.

Here's what it is and why it is useful: YQL data can be returned in XML which is annoying to use in JavaScript (for starters because of crossdomain issues in Ajax). JSON is much easier as it is native to JavaScript. JSON-P makes it even more easy for us to use as JSON data wrapped in a function call allows us to get the data by creating script nodes dynamically.

Where it falls apart is when you want to get back HTML from some system (not on your own server) and use it in JavaScript. You either need to convert the XML to JavaScript and create HTML elements from it or you need to loop through a JSON construct and assemble HTML from it. JSONP-X works around that step for you. In essence it is XML as a string returned inside a JSON object.

XML:

JSON:

JSON-P:

JSONP-X:

The way to invoke the JSONP-X output is to provide a format parameter of xml and a callback parameter.

This allows me for example to get the list of people I follow on twitter from my twitter homepage and display it in another document with a few lines of JavaScript without any need of using the API or having a local proxy:

More details on this are available in this blog post.

Alex MacCaw has released Machsend, a Yahoo! Browser Plus plugin that enables P2P file transfers from inside the browser.

It showcases what can be done with a BP plugin, leaving you wish cross browser functionality.

I guess it is kinda fun to hack the browser :)

The annoying thing is that Placemaker only support POST request and does not have a JSON output - for now. GeoMaker allows non-developers to enter some text or a URL, filter the results (using YUI datatable) to remove false positives (no system is perfect) and get the embed code for a Yahoo Map or a list of microformatted locations as copy+paste. See the screencast to get the end user experience:

Of course, every time you build something like that, red-blooded developers will ask for an API to do the same thing (and pointing them to Placemaker wasn't enough). So here it is:

http://icant.co.uk/geomaker/api.php takes two parameters: url of the web document to load and output which could be map, kml, microformats, csv, or json (with callback for JSON-P). Using this you can analyze a url in JavaScript and get the data back as an array:

The Ajaxian.com output right now would be:

We’ve spent a lot of time in this release cycle refining the core elements of YUI 3 — YUI, Node, and Event — to ensure that we have the right API going forward. Performance is improved, and we’ve refined our module/submodule structure. In some cases we’ve added significant new features, including intrinsic support for event delegation in the Event package; however, the focus is on moving the base library to production quality.

Several YUI 2.x components make their YUI 3 debut in this release:

1. DataSource: YUI’s data abstraction layer provides a standard interface into data sets, regardless of the data’s origin (local, XHR, XSS, etc.) and format (JSON, XML, CSV, etc.);
2. ImageLoader: ImageLoader allows you to defer the loading of images that aren’t in the viewport when the page paints, throttling bandwidth usage and improving performance;
3. History: The History component gives you control of the brower’s back button within the context of a single-page web application;
4. StyleSheet: StyleSheet makes it easy to create and modify CSS rules on the fly, allowing you to dynamically style page elements with fewer repaints.

As part of the more granular packaging in the new codeline, we’ve created separate YUI 3 modules to house functionality that in YUI 2 was bundled with other components. Cache, DataType and DataSchema debut in this release; each of these used to be a part of DataSource.

To keep up to date, check out the roadmap and dashboard that the team keeps up to date.

There are many geo lookup tools and a lot of (at least UK) web sites started doing post code lookup tools. This is cool, but as annoying when I spent another reload of a full large form for a postcode lookup that went wrong. With the UK being quite a mess when it comes to post codes this is not uncommon.

Now, to work around these issues I've put together a small JavaScript wrapper library called GeoFill (also available on GitHub). This one makes it easy to provide a "is this your location" functionality in JavaScript:

GeoFill has two different methods to automatically fill forms with geo information:

• geofill.find(properties) does an IP lookup of the current user and tries to get the geographical data from that one.
• geofill.lookup(properties,postcode) tries to get the geographical data from the postcode provided in your form.

Both methods can either automatically fill a form when you provide them with an object telling it which info should go into the form with a given ID, but also have a callback property that allows you to use the data immediately

The returned object is:

The find method uses the GeoIP lookup:

And returns (in my case):

What most of these talks and videos have in common is application of JavaScript and how to deal with issues that will occur. Information about designing and architecting large JavaScript solutions on the other hand are rare. Satyen Desai now broke the silence and gave a great talk on the architecture and design decisions of the the third generation of the Yahoo User Interface library:

• Watch a larger resolution version and read the transcripts
• download the slides

Try it out here and you see what I mean.

The next step of this interface was to open it out to the public. You can define an "Open Table" as a simple XML schema and bring your own API into this interface with that.

One thing that's been burning on my tongue to tell the world about has been finally released now: YQL execute. Instead of making the YQL language itself much more complex (and thus running in circles) we now allow you to embed JavaScript in the Open Table XML that will run on the YQL server and allow you to access other web services, authenticate and scrape HTML with JavaScript and E4X. As Simon Willison put it:

This is nuts (in a good way). Yahoo!’s intriguing universal SQL-style XML/JSONP web service interface now supports JavaScript as a kind of stored procedure language, meaning you can use JavaScript and E4X to screen-scrape web pages, then query the results with YQL.

Using this, you can augment the original functionality of YQL to whatever you need. For example, you can scrape HTML with YQL using XPATH, but there was no way to use CSS selectors. Using an open table that invokes James Padolsey's css2xpath JavaScript on the server side, this is now possible.

Run this query in YQL

The data table is pretty easy:

Check the official Yahoo Developer Network blog post on YQL execute for more examples, including authentication examples for flickr and netflix.

• Use conditional GET requests
• Retain the most recent Etag and Last-Modified date of the polled resource
• Disable polling when the browser window is inactive

Implementing a smart polling process in our application’s rich UI gives us some desired benefits:

• Removal of the refresh button
• Automatic updating of the UI when the resource on the server has changed
• Less repainting of the page since the DOM is touched only when the data has changed
• Changes to the UI only happen when the window is active (the user sees them) as polling is paused while the user is doing something else

The component that Eric wrote means that you can set of a poller like this:

The main findings of the team were that eval() is not only evil but also very slow whereas dynamic script nodes are fast but insecure. The solution was to do a custom evaluation of string data rather than using JSON:

Having set the performance bar pretty high with the last approach, we dove into custom data formats. The challenge would be to create a format that we could parse ourselves, using JavaScript’s String and RegExp methods, that would also match the speed of JSON executed natively. This would allow us to use Ajax again, but keep the data restricted to our domain.

Since we had already discovered that some methods of string manipulation didn’t perform well on large strings, we restricted ourselves to a method that we knew to be fast: split(). We used control characters to delimit each contact, and a different control character to delimit the fields within each contact. This allowed us to parse the string into contact objects with one split, then loop through that array and split again on each string.

Once this had been speeded up, all they needed to use was the YUI AutoComplete control and voilà - fast client side searches even with massive datasets.

Great news for YUI3: Matt Sweeney, a YUI Developer, has integrated the Sizzle JavaScript Selector Library in a branch of the YUI3 code-base on GitHub. There has been interest from the community about this integration for quite some time. I personally was hoping for Sizzle adoption into YUI3; this now appears to be the case.

Using the Selector utility in YUI2 I would find myself hitting bugs and roadblocks where my expectations weren’t matching the outcomes. I’ve felt since first using the selector engine in YUI that it didn’t compare in speed, robustness, or completeness to jQuery’s. With the Sizzle project, there’s a narrow focus and distinct vision: To make the best damn selector engine. Since a selector engine is an essential component to any modern JavaScript library, why not make one really good one for all libraries to use? That’s the route the Sizzle project has taken.

Once Sweeney’s branch is merged into the YUI3 Master Head, I’ll be doing a git pull and give it some exercise.

The YUI Library turned three years old this month and we’d like to invite you — our community of developers and implementers — to come out and celebrate! To celebrate our third birthday (and our 2.7.0 release), we’ll be hosting a recession-chic happy hour at the Blue Chalk Cafe in downtown Palo Alto, Thursday, February 26 from 6:00 to 8:00 pm. Nothing too fancy, but we’ll have fun goodies to give away and the first few rounds of drinks are on us (until the tab runs out). Details and signups are at Upcoming. Hope to see you there!

Should prove to be lots of fun.

But that's not all! You'll have plenty to talk about at the happy hour as the team just released YUI 2.7.0:

Version 2.7.0 introduces a new StyleSheet component, graduates three components out of “beta”, improves support for the upcoming release of IE8, includes over 180 bug fixes and enhancements, and ships with more than 300 functional examples.

New features include:

• The YUI StyleSheet Utility
• Enhancements to the Charts control
• Updates to both the Container & Treeview components
• Internet Explorer 8 Support
• New components promoted out of “beta” status:
  • Element
  • ProfilerViewer
  • Selector

In addition, they produced over 180 Enhancements and Bug Fixes.

One nice example of the power of Microformats and YQL is George Ornbo's jQuery script that allows you to pre-fill forms with data retrieved from hcards:

I've written similar tests before but soon gave up on trying to parse microformats by hand. With YQL doing that job for us the practical application of microformatted data became a lot easier, so let's go wild.