As preparation for an upcoming tech talk about Placemaker I thought it would be good to take a bit of the pain out of the geolocation service by making an interface for it. Placemaker works the following way: you post some content or a URL to it, it goes through the content or gets the content from the URL and analyzes it. It then finds geographical locations in the text and disambiguates them (for example to skip names like "Jack London" and not consider it the city London). Finally you get it back as XML.
The annoying thing is that Placemaker only support POST request and does not have a JSON output - for now. GeoMaker allows non-developers to enter some text or a URL, filter the results (using YUI datatable) to remove false positives (no system is perfect) and get the embed code for a Yahoo Map or a list of microformatted locations as copy+paste. See the screencast to get the end user experience:
Of course, every time you build something like that, red-blooded developers will ask for an API to do the same thing (and pointing them to Placemaker wasn't enough). So here it is:
http://icant.co.uk/geomaker/api.php takes two parameters: url of the web document to load and output which could be map, kml, microformats, csv, or json (with callback for JSON-P). Using this you can analyze a url in JavaScript and get the data back as an array:
We’ve spent a lot of time in this release cycle refining the core elements of YUI 3 — YUI, Node, and Event — to ensure that we have the right API going forward. Performance is improved, and we’ve refined our module/submodule structure. In some cases we’ve added significant new features, including intrinsic support for event delegation in the Event package; however, the focus is on moving the base library to production quality.
Several YUI 2.x components make their YUI 3 debut in this release:
DataSource: YUI’s data abstraction layer provides a standard interface into data sets, regardless of the data’s origin (local, XHR, XSS, etc.) and format (JSON, XML, CSV, etc.);
ImageLoader: ImageLoader allows you to defer the loading of images that aren’t in the viewport when the page paints, throttling bandwidth usage and improving performance;
History: The History component gives you control of the brower’s back button within the context of a single-page web application;
StyleSheet: StyleSheet makes it easy to create and modify CSS rules on the fly, allowing you to dynamically style page elements with fewer repaints.
As part of the more granular packaging in the new codeline, we’ve created separate YUI 3 modules to house functionality that in YUI 2 was bundled with other components. Cache, DataType and DataSchema debut in this release; each of these used to be a part of DataSource.
To keep up to date, check out the roadmap and dashboard that the team keeps up to date.
One thing that annoys me is that in the days of geolocation and location recognition I still have to enter my country and city in forms over and over again. This is especially annoying when you like in the UK which could be anywhere in these wonderful large dropdowns (UK, England, Great Britain, United Kingdom, Blighty...).
There are many geo lookup tools and a lot of (at least UK) web sites started doing post code lookup tools. This is cool, but as annoying when I spent another reload of a full large form for a postcode lookup that went wrong. With the UK being quite a mess when it comes to post codes this is not uncommon.
Now, to work around these issues I've put together a small JavaScript wrapper library called GeoFill (also available on GitHub). This one makes it easy to provide a "is this your location" functionality in JavaScript:
GeoFill has two different methods to automatically fill forms with geo information:
geofill.find(properties) does an IP lookup of the current user and tries to get the geographical data from that one.
geofill.lookup(properties,postcode) tries to get the geographical data from the postcode provided in your form.
Both methods can either automatically fill a form when you provide them with an object telling it which info should go into the form with a given ID, but also have a callback property that allows you to use the data immediately
Talks about JavaScript are not a rare occasion any more, everybody has something to say about this wonderfully versatile language. Douglas Crockford taught us a lot about the language itself, John Resig and Peter Paul Koch taught us how browsers deal (and fail dealing) with it and there are dozens of screencasts of how to use this or that language.
What most of these talks and videos have in common is application of JavaScript and how to deal with issues that will occur. Information about designing and architecting large JavaScript solutions on the other hand are rare. Satyen Desai now broke the silence and gave a great talk on the architecture and design decisions of the the third generation of the Yahoo User Interface library:
I am a big fan of YQL, a terribly easy and fuss-free way to access APIs and mix data retrieved from them in a simple, SQL style language. Say for example you want photos of Paris,France from Flickr that are licensed with Creative Commons attribution, you can do this with a single command:
select * from flickr.photos.info where photo_id in (select id from flickr.photos.search where woe_id in (select woeid from geo.places where text='paris,france') and license=4)
One thing that's been burning on my tongue to tell the world about has been finally released now: YQL execute. Instead of making the YQL language itself much more complex (and thus running in circles) we now allow you to embed JavaScript in the Open Table XML that will run on the YQL server and allow you to access other web services, authenticate and scrape HTML with JavaScript and E4X. As Simon Willison put it:
This is nuts (in a good way). Yahoo!’s intriguing universal SQL-style XML/JSONP web service interface now supports JavaScript as a kind of stored procedure language, meaning you can use JavaScript and E4X to screen-scrape web pages, then query the results with YQL.
Using this, you can augment the original functionality of YQL to whatever you need. For example, you can scrape HTML with YQL using XPATH, but there was no way to use CSS selectors. Using an open table that invokes James Padolsey's css2xpath JavaScript on the server side, this is now possible.
use 'http://yqlblog.net/samples/data.html.cssselect.xml' as data.html.cssselect; select * from data.html.cssselect where url="www.yahoo.com" and css="#news a"
The main findings of the team were that eval() is not only evil but also very slow whereas dynamic script nodes are fast but insecure. The solution was to do a custom evaluation of string data rather than using JSON:
Having set the performance bar pretty high with the last approach, we dove into custom data formats. The challenge would be to create a format that we could parse ourselves, using JavaScript’s String and RegExp methods, that would also match the speed of JSON executed natively. This would allow us to use Ajax again, but keep the data restricted to our domain.
Since we had already discovered that some methods of string manipulation didn’t perform well on large strings, we restricted ourselves to a method that we knew to be fast: split(). We used control characters to delimit each contact, and a different control character to delimit the fields within each contact. This allowed us to parse the string into contact objects with one split, then loop through that array and split again on each string.
for(var n = 0, len = that.contacts.length, contactSplit; n <len; n++){
contactSplit = that.contacts[n].split("\\a");
that.contacts[n] = {};
that.contacts[n].n = contactSplit[0];
that.contacts[n].e = contactSplit[1];
that.contacts[n].u = contactSplit[2];
that.contacts[n].r = contactSplit[3];
that.contacts[n].s = contactSplit[4];
that.contacts[n].f = contactSplit[5];
that.contacts[n].a = contactSplit[6];
that.contacts[n].d = contactSplit[7];
that.contacts[n].y = contactSplit[8];
}
Once this had been speeded up, all they needed to use was the YUI AutoComplete control and voilà - fast client side searches even with massive datasets.
Using the Selector utility in YUI2 I would find myself hitting bugs and roadblocks where my expectations weren’t matching the outcomes. I’ve felt since first using the selector engine in YUI that it didn’t compare in speed, robustness, or completeness to jQuery’s. With the Sizzle project, there’s a narrow focus and distinct vision: To make the best damn selector engine. Since a selector engine is an essential component to any modern JavaScript library, why not make one really good one for all libraries to use? That’s the route the Sizzle project has taken.
Once Sweeney’s branch is merged into the YUI3 Master Head, I’ll be doing a git pull and give it some exercise.
Some very cool news out of the YUI project as their namesake library turned 3 years old yesterday (That's like 25 in internet years, right?). The framework has really grown-up to provide a solid foundation on which to build upon and developers are certainly benefiting from it's excellent documentation and QA. To celebrate YUI's birthday, the team will be hosting a happy hour:
The YUI Library turned three years old this month and we’d like to invite you — our community of developers and implementers — to come out and celebrate! To celebrate our third birthday (and our 2.7.0 release), we’ll be hosting a recession-chic happy hour at the Blue Chalk Cafe in downtown Palo Alto, Thursday, February 26 from 6:00 to 8:00 pm. Nothing too fancy, but we’ll have fun goodies to give away and the first few rounds of drinks are on us (until the tab runs out). Details and signups are at Upcoming. Hope to see you there!
Should prove to be lots of fun.
But that's not all! You'll have plenty to talk about at the happy hour as the team just released YUI 2.7.0:
Version 2.7.0 introduces a new StyleSheet component, graduates three components out of “beta”, improves support for the upcoming release of IE8, includes over 180 bug fixes and enhancements, and ships with more than 300 functional examples.
Ever since YQL started supporting microformats scanning (the outcome of having a coffee with the team last time I visited the main office) I tried to find good examples of how you can use this. Microformats rock, we all know that but so far I am still waiting for the killer application or implementation for it. The wiki examples of microformats are getting stale indeed and in general I start to wonder what it needs to make the main market understand the benefits and implement them.
One nice example of the power of Microformats and YQL is George Ornbo's jQuery script that allows you to pre-fill forms with data retrieved from hcards:
I've written similar tests before but soon gave up on trying to parse microformats by hand. With YQL doing that job for us the practical application of microformatted data became a lot easier, so let's go wild.
Stuart Colville has announced a fun little project that he and fellow collaborators Ed Elliot and Cyril Doussin have worked on.
Their Favicon Generator lets you upload an image and then go in and tweak the pixels :)
Although many modern web browsers support favicons saved as GIFs, PNGs or other popular file formats all versions of Internet Explorer still require favicons to be saved as ICO files (a Microsoft icon format). This tool provides an easy way to convert any GIF, PNG or JPEG to ICO which is supported by all modern web browsers. It also enables you to create favicons from scratch via a handy online editor. Additionally the editor lets you manually tweak generated favicons to ensure the best possible result.
The main difference with the tool is that we’ve provided an editor which allows you to tweak the generated icons which is often necessary particularly if you’re creating a favicon from a much larger graphic.
The post discusses how you can use the loader for your own libraries, giving you full dependency management, similar to that of the Dojo loader and others:
As YUI comes with a special module for loading dependencies, the YUI Loader Utility, we explored if we could use YUI for our needs. The Loader’s feature list reads like our wish-list:
Reliable, sorted loading of dependencies
Safe, efficient mechanism for adding new components to a page on which YUI may already be present.
Automatic use of rolled-up files.
So we’ve developed our own solution on top of the YUI loader for our Framework. It consists of two components: A Compile-time dependency analyzer and a configuration file generated by the dependency analyzer that tells YUI how to load our modules.
Then we get to see the details on how you take this API:
JSLint Multi uses Douglas Crockford’s excellent JSLint tool to check your Javascript files for potential problems.
Drag a folder unto JSLint Multi, and it will give you an overview of the JSLint status of every Javascript file in that folder and folders below it. Click a file in the list, or drag a file, URI or some Javascript code unto the widget, and it will displays details of the first problem found. Edit the file, and JSLint Multi will instantly recheck it! It even picks up new files added to the monitored folder!
From the post, it seems the YUI team feels that moving to Git will make the process of external contributions easier and considering how everyone raves about Git, I can see the rationale behind it.