This loads the document any link with a class of "ajaxtrigger" points to and updates the content of the element with the ID "target". If the link is a third party link on another domain it fails though - and silently at that. Normally you'd work around that with a server-side proxy, but you can actually do without it.
YQL is a hosted web service that can scrape HTML for you. It also runs the HTML through HTML Tidy and caches it for you. For example to load http://bbc.co.uk you'd use the following statement:
You could request JSON-P by setting the output format to json and define a callback, but that would give the HTML back as a massive object which is not nice. YQL also offers JSON-P-X as an alternative which is a JSON object with a callback and the HTML as a string inside a simple Array. See it by clicking the following URL:
Now, using jQuery's getJSON() we can load this even without a named callback function. That way we can use one method for content that is third party and simply use load() for the other:
Lloyd Hilaiel of Yahoo! BrowserPlus fame, has a little holiday gift for us. He has created a simple library called Orderly that "is a textual format for describing JSON. Orderly can be compiled into JSONSchema. It is designed to be easy to read and write."
JSON Hyper Schemas can be referenced from instances by Link headers or media type parameters. A simple example illustrates how JSON properties have universally locatable definitions:
This message gives the authoritative URI for the schema. With a look at the schema, we can see how each property has a corresponding definition with an authoritative URI as well:
"title": {"type": "string", "description": "The title of the book"},
"price": {"type": "number", "description": "The price of the book in US"},
}
}
JSON Hyper Schema is a sufficient building block upon which we can realize the goals of namespacing, namely, meanings from distributed parties can be authoritatively ascribed to elements of a data structure with property name sharing and conflict resolution control. What is elegant about this approach is that basically nothing needs to be modified in the actual instance data structures themselves. URI attribution is done through media type parameters in combination with schemas. The actual data stays unchanged, it is still the original simple, easy to read, compact JSON. JSON namespacing not only fits with JSON technically, but it fits in spirit: data is simple and light.
An interesting piece by Neil Fraser shows that using JSON-P with generated script nodes can be quite a memory leak. Normally you'd add information returned from an API in JSON-P with a generated script node:
// Add the script to the head, upon which it will load and execute.
var head = document.getElementsByTagName('head')[0];
head.appendChild(script)
The issue there is that you clog up the head of the document with lots of script nodes, which is why most libraries (like the YUI get() utility) will add an ID to the script element and remove the node after successfully retrieving the JSON data:
The issue with this is that browsers do remove the node but fail to do a clean garbage collection of the JavaScript at the same time. This means to cleanly remove the script and its content, you need to also delete the properties of the script by hand:
Here's what it is and why it is useful: YQL data can be returned in XML which is annoying to use in JavaScript (for starters because of crossdomain issues in Ajax). JSON is much easier as it is native to JavaScript. JSON-P makes it even more easy for us to use as JSON data wrapped in a function call allows us to get the data by creating script nodes dynamically.
Where it falls apart is when you want to get back HTML from some system (not on your own server) and use it in JavaScript. You either need to convert the XML to JavaScript and create HTML elements from it or you need to loop through a JSON construct and assemble HTML from it. JSONP-X works around that step for you. In essence it is XML as a string returned inside a JSON object.
The way to invoke the JSONP-X output is to provide a format parameter of xml and a callback parameter.
This allows me for example to get the list of people I follow on twitter from my twitter homepage and display it in another document with a few lines of JavaScript without any need of using the API or having a local proxy:
As preparation for an upcoming tech talk about Placemaker I thought it would be good to take a bit of the pain out of the geolocation service by making an interface for it. Placemaker works the following way: you post some content or a URL to it, it goes through the content or gets the content from the URL and analyzes it. It then finds geographical locations in the text and disambiguates them (for example to skip names like "Jack London" and not consider it the city London). Finally you get it back as XML.
The annoying thing is that Placemaker only support POST request and does not have a JSON output - for now. GeoMaker allows non-developers to enter some text or a URL, filter the results (using YUI datatable) to remove false positives (no system is perfect) and get the embed code for a Yahoo Map or a list of microformatted locations as copy+paste. See the screencast to get the end user experience:
Of course, every time you build something like that, red-blooded developers will ask for an API to do the same thing (and pointing them to Placemaker wasn't enough). So here it is:
http://icant.co.uk/geomaker/api.php takes two parameters: url of the web document to load and output which could be map, kml, microformats, csv, or json (with callback for JSON-P). Using this you can analyze a url in JavaScript and get the data back as an array:
It includes compatibility tables, and will try to run the tests on your browser to give you feedback. It also includes sample code to check web browser support that you can use in your own projects.
When you provide a JSON output for developers it does make sense to also allow for a callback parameter. That way your code can be used in script nodes without having to use any backend at all. Commonly this is called JSON-P and has been covered here in the long long ago. One of the issues with JSON-P is that when the callback method is not defined it throws an error.
The Bing API is the first instance where I have seen that they worked around that as the output is this:
I have no clue what the /* pageview_candidate */ is about and frown upon omitting the {} of the if statement, but I must say I do like this. One issue is that while end users don't get annoyed with errors, developers don't have a clue what happened either as the error is silent. One proposal would be to use a console.log() when there is an error:
The latest beta of Persevere features a new native object storage engine called JavaScriptDB that provides high-end scalability and performance. Persevere now outperforms the common PHP and MySQL combination for accessing data via HTTP by about 40% and outperforms CouchDB by 249%. The new storage engine is designed and optimized specifically for persisting JavaScript and JSON data with dynamic object structures. It is also built for extreme scalability, with support for up to 9,000 petabytes of JSON/JS data in addition to any binary data.
This comparison isn't exactly apples-to-apples as it turns out–for the web app use case, Perservere has a bunch of value-adds on top of data storage:
Persevere/JavaScriptDB goes further [than relational DBs] with the flexibility to evolve schemas and handle partial schemas. Persevere also provides integrated server side JavaScript (SSJS) with persistence, Comet-driven data change notifications, JSONQuery, standards based HTTP interface with content negotiation, JSON-RPC interface to SSJS, cross-domain handling, CSRF protection, and more. All of these things are additional features that one would have to add to the stack for other storage systems, making them even slower. Persevere includes this functionality out of the box, while still maintaining extremely fast performance.
Kris spends a bit of time in his post explaining his test setup, but then gets to the good stuff:
So how does Persevere achieve this level of performance with the JavaScriptDB storage? The dynamic object-oriented nature of the data that is stored in JavaScriptDB is much different than that of a traditional relational database, so a number of innovative approaches were employed.
He goes into quite a bit of detail explaining the implementation details behind JavaScriptDB. The summary (with lightly edited quotes) is:
Direct Data-Bound Object Representation: "In a traditional application stack, a record must have separate in-memory representations for [the database] and [the application] result set which then might be mapped to an object representation. With JavaScriptDB, the single in-memory object is reused for all result sets and data caching."
Shared Cache of Objects with Copy-on-Write
Append-based Database Storage: "Many traditional database commit data to a transaction log before committing data to the table, requiring multiple writes. JavaScriptDB appends transactional data directly to the main storage file; writes can be committed with a single IO operation."
BrowserCouch is an attempt at an in-browser MapReduce implementation. It's written entirely in JavaScript and intended to work on all browsers, gracefully upgrading when support for better efficiency or feature set is detected.
Not coincidentally, this library is intended to mimic the functionality of CouchDB on the client-side, and may even support integration with CouchDB in the future.
Why?
This prototype is intended as a response to Vladimir Vuki?evi?'s blog post entitled HTML5 Web Storage and SQL. A CouchDB-like API seems like a nice solution to persistent storage on the Web because so many of its semantics are delegated out to the JavaScript language, which makes it potentially easy to standardize. Furthermore, the MapReduce paradigm also naturally takes advantage of multiple processor cores—something that is increasingly common in today's computing devices.
There is also an interactive CouchDB project that "is an emulator written in 100% JavaScript with tons of jQuery thrown in. It also implements the collation schemes as well as the map/reduce algorithms. While it doesn’t demonstrate replication, conflict management and a host of other capabilities in CouchDB, it does strive to illustrate concepts like schema-less JSON documents, map/reduce and how these things fit together."
When Aaron talked about the Chrome extension API he mentioned how he would see if JSON Schema could help them have a JSON heavy API and allow them to easily validate.
He has quickly reported back and is happy with the results.
The main findings of the team were that eval() is not only evil but also very slow whereas dynamic script nodes are fast but insecure. The solution was to do a custom evaluation of string data rather than using JSON:
Having set the performance bar pretty high with the last approach, we dove into custom data formats. The challenge would be to create a format that we could parse ourselves, using JavaScript’s String and RegExp methods, that would also match the speed of JSON executed natively. This would allow us to use Ajax again, but keep the data restricted to our domain.
Since we had already discovered that some methods of string manipulation didn’t perform well on large strings, we restricted ourselves to a method that we knew to be fast: split(). We used control characters to delimit each contact, and a different control character to delimit the fields within each contact. This allowed us to parse the string into contact objects with one split, then loop through that array and split again on each string.
for(var n = 0, len = that.contacts.length, contactSplit; n <len; n++){
contactSplit = that.contacts[n].split("\\a");
that.contacts[n] = {};
that.contacts[n].n = contactSplit[0];
that.contacts[n].e = contactSplit[1];
that.contacts[n].u = contactSplit[2];
that.contacts[n].r = contactSplit[3];
that.contacts[n].s = contactSplit[4];
that.contacts[n].f = contactSplit[5];
that.contacts[n].a = contactSplit[6];
that.contacts[n].d = contactSplit[7];
that.contacts[n].y = contactSplit[8];
}
Once this had been speeded up, all they needed to use was the YUI AutoComplete control and voilà - fast client side searches even with massive datasets.
JSONView is a new Firefox extension that gives you a nice way to view your JSON documents (JSONovich also does the trick).
Ben Hollis talks about the extension:
The extension itself is pretty simple. I wasn’t sure how to approach the problem of supporting a new content type for Firefox, so I followed the example of the wmlbrowser extension and implemented a custom nsIStreamConverter. What this means is that I created a new component that tells Firefox “I know how to translate documents of type application/json into HTML”. And that it does - parsing the JSON using the new native JSON support in Firefox 3 (for speed and security) and then constructing an HTML document that it passes along the chain. This seems to work pretty well, though there are some problems - some parts of Firefox forget the original type of the document and treat it as HTML, so “View Page Info” reports “text/html” instead of “application/json”, “Save as…” saves the generated HTML, Firebug sees the generated HTML, etc. Just recently I came across the nsIURLContentListener interface, which might offer a better way of implementing JSONView, but I’m honestly not sure - the Mozilla documentation is pretty sparse and it was hard enough to get as far as I did. I’m hoping some Mozilla gurus can give me some pointers now that it’s out in the open.
The native JSON API is part of the upcoming 3.1 revision of ECMAScript, so we should see it adopted in browsers pretty quickly. It’s also API compatible with json2.js, as you note, so many many web users will get the performance win without apps needing to update.
I suspect that the performance advantage for native JSON is even more pronounced on the encoding side, but I don’t have tests to hand to back it up.
This was a comment by Mike Shaver, VP of Engineering at Mozilla (disclaimer: where I work).
In case you haven’t heard, one of Firefox 3.1’s awesome new features will be native JSON support. This is totally sweet for two reasons:
eval’ing JSON in the browser is unsafe. Using native JSON parsing protects you against possible code execution.
Safely eval’ing JSON with a 3rd party library can be orders of magnitude slower. Native JSON parsing is much faster.
How does native JSON work compared to plain old eval? Simple:
var jsonString = '{"name":"Ryan", "address":"Mountain View, CA"}';
var person = JSON.parse(jsonString);
// 'person' is now a JavaScript object with 2 properties; name and address
Pretty easy huh? And here’s how to get a JSON string from an object:
var personString = JSON.stringify(person);
// 'personString' now holds the string '{"name":"Ryan", "address":"Mountain View, CA"}'
“But wait!”, you say. “How is it safer? How much faster is it compared to eval?”. Ok, I’ll show you.
Gotta love it when browsers implement something that just starts to make apps faster without potentially knowing about it.
An almost non-lossy serialization format for sending XML as JSON (plain text in between elements is ignored). Uses the (element-name, attribute-dictionary, list-of-children) tuple format, which sadly means many common cases end up taking more bytes than the original XML. Still an improvement on serializations that behave differently when a list of children has only one item in it.
Have you seen this path before? Hating on XML and going for JSON, but then trying to JSON as XML and at the end of it you just s/</{/g.
We don't often post about general APIs. I let John Musser handle that on Programmable Web, but this one strikes a chord with me.
Kiva, the distributed micro loan platform, has released a new developer API that gives third parties access to create innovative applications on top of the platform:
The initial release of the Kiva API is concerned with helping you find and present public data from Kiva. We want to help you find things like loans, lenders, and partners and help you represent those in your applications and products. We think there are a lot of really useful things that can be done just by helping the Kiva community find information in different ways or experience it in a new context. Moreover, understanding our core set of data is the most important building block to doing more complex operations in future versions of the API.
I have just started poking at the data available, as I suspect network analysis will be able to help predict rates of return.
When I knew that Skylar Woodward was in on the action I knew something cool was coming. The API so far is typically JSON and RESTful (and XML if you haaave too). I wonder if anyone has written a nice JavaScript wrapper around the API yet?
