It was pretty simple to insert a step to save the data to Jdrop. The bookmarklet’s display code is easily re-used by wrapping the data in JSON and passing it back to the display code inside Jdrop’s web page. That, in a nutshell, is Jdrop.

Meanwhile, Thomas Fuchs has added Jdrop to his DOM Monster bookmarklet. JDrop is alpha mode, reminds Souders.

The first and less visible of the two is in the JsonSerializer. Because there is no static list of fields or properties for a dynamic type the serializer interrogates the value for its members prior to serializing and deserializing. The end result is serializing should Just Work for any type that implements IDynamicMetaObjectProvider.

… I think the Web community has spoken, and it’s clear that what it wants is HTML5, JavaScript and JSON. XML isn’t going away but I see it being less and less a Web technology; it won’t be something that you send over the wire on the public Web, but just one of many technologies that are used on the server to manage and generate what you do send over the wire.

And he continues:

…In the short-term, I think the challenge is how to make HTML5 play more nicely with XML. In the longer term, I think the challenge is how to use our collective experience from building the XML stack to create technologies that work natively with HTML, JSON and JavaScript, and that bring to the broader Web developer community some of the good aspects of the modern XML development experience.

XML vs the Web – Clark’s Random Thoughts blog

Any data, anywhere.

JSonduit is a service that can turn practically anything on the web into a JSON feed that any website may consume. A JSON conduit, if you will.

Feeds are created by specifying one or more source URLs and a custom transform, written in JavaScript, that can manipulate the data before the feed is served.

JSonduit also provides a hosting service for web widgets so that any site can easily display JSonduit feeds on their pages. In fact, those recent/popular lists of you see below are actual widgets served by the JSonduit service; all done in a couple of lines of JavaScript (go ahead, view the page source).

To see what it is like to query the world in this manner, check out the most popular feed.... a view on hacker news:

This returns something like:

If this looks a touch familiar though... remember that the awesome YQL gives you acccess to the Web from a simple query language too.

If you are into switches and Erlang, it will look maybe a bit familiar, with entries such as:

More details:

TeleHash enables applications to connect directly and participate as servers on the edge of the network. It is designed to efficiently route and distribute small bits of data in order for applications to discover each other directly or in relation to events around piece of shared content. The core benefits of TeleHash over other similar platforms and protocols is that it is both generic (not tied to any specific application or content structures) and is radically decentralized with no servers or points of central control.

It works by sending and receiving very simple small bits of JSON via UDP using an easy routing system based on Kademlia, a proven and popular Distributed Hash Table. Everything within TeleHash is routed based on a generic SHA hash, usually of something specific to an application or something common like a URL.

Wow :)

This loads the document any link with a class of "ajaxtrigger" points to and updates the content of the element with the ID "target". If the link is a third party link on another domain it fails though - and silently at that. Normally you'd work around that with a server-side proxy, but you can actually do without it.

YQL is a hosted web service that can scrape HTML for you. It also runs the HTML through HTML Tidy and caches it for you. For example to load http://bbc.co.uk you'd use the following statement:

As a URL this turns into:

You could request JSON-P by setting the output format to json and define a callback, but that would give the HTML back as a massive object which is not nice. YQL also offers JSON-P-X as an alternative which is a JSON object with a callback and the HTML as a string inside a simple Array. See it by clicking the following URL:

http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%27http%3A%2F%2Fbbc.co.uk%27%0A&format=xml&diagnostics=false&callback=foo

Now, using jQuery's getJSON() we can load this even without a named callback function. That way we can use one method for content that is third party and simply use load() for the other:

You can see the demo in action here, more details are available on my blog and the source is available on GitHub.

He shares:

A little bit of orderly...

PLAIN TEXT

JAVASCRIPT:

1.  
2. object {
3.   string name;
4.   string description?;
5.   string homepage /^http:/;
6.   integer {1500,3000} invented;
7. }*;
8.  

...describes a little bit of JSON...

PLAIN TEXT

JAVASCRIPT:

1.  
2. {
3.   "name": "orderly",
4.   "description": "A schema language for JSON",
5.   "homepage": "http://orderly-json.org",
6.   "invented": 2009
7. } 
8.  

...and compiles into JSONSchema.

PLAIN TEXT

JAVASCRIPT:

1.  
2. {
3.   "type": "object",
4.   "properties": {
5.     "name": {
6.       "type": "string"
7.     },
8.     "description": {
9.       "type": "string",
10.       "optional": true
11.     },
12.     "homepage": {
13.       "type": "string",
14.       "pattern": "^http:"
15.     },
16.     "invented": {
17.       "type": "integer",
18.       "minimum": 1500,
19.       "maximum": 3000
20.     }
21.   },
22.   "additionalProperties": true
23. }
24.  

Go into detail on Orderly in the documentation or try the
interactive by-directional Orderly - JSONSchema area.

In JSON namespacing he discusses JSON Hyper Schema which aims to cure JSON from XML namespacing colon cancer.

JSON Hyper Schemas can be referenced from instances by Link headers or media type parameters. A simple example illustrates how JSON properties have universally locatable definitions:

PLAIN TEXT

JAVASCRIPT:

1.  
2. Content-Type: application/json; schema=http://www.book-warehouse.com/book-schema
3. [
4.   {"title": "Oliver Twist", "price": 16.99},
5.   {"title": "Robinson Crusoe", "price": 15.99}
6. ]
7.  

This message gives the authoritative URI for the schema. With a look at the schema, we can see how each property has a corresponding definition with an authoritative URI as well:

PLAIN TEXT

JAVASCRIPT:

1.  
2. Content-Type: application/schema+json; schema=http://json-schema.org/hyper-schema
3. {
4.   "properties": {
5.      "title": {"type": "string", "description": "The title of the book"},
6.      "price": {"type": "number", "description": "The price of the book in US"},
7.   }
8. }
9.  

JSON Hyper Schema is a sufficient building block upon which we can realize the goals of namespacing, namely, meanings from distributed parties can be authoritatively ascribed to elements of a data structure with property name sharing and conflict resolution control. What is elegant about this approach is that basically nothing needs to be modified in the actual instance data structures themselves. URI attribution is done through media type parameters in combination with schemas. The actual data stays unchanged, it is still the original simple, easy to read, compact JSON. JSON namespacing not only fits with JSON technically, but it fits in spirit: data is simple and light.

The issue there is that you clog up the head of the document with lots of script nodes, which is why most libraries (like the YUI get() utility) will add an ID to the script element and remove the node after successfully retrieving the JSON data:

The issue with this is that browsers do remove the node but fail to do a clean garbage collection of the JavaScript at the same time. This means to cleanly remove the script and its content, you need to also delete the properties of the script by hand:

Read the whole post here

Here's what it is and why it is useful: YQL data can be returned in XML which is annoying to use in JavaScript (for starters because of crossdomain issues in Ajax). JSON is much easier as it is native to JavaScript. JSON-P makes it even more easy for us to use as JSON data wrapped in a function call allows us to get the data by creating script nodes dynamically.

Where it falls apart is when you want to get back HTML from some system (not on your own server) and use it in JavaScript. You either need to convert the XML to JavaScript and create HTML elements from it or you need to loop through a JSON construct and assemble HTML from it. JSONP-X works around that step for you. In essence it is XML as a string returned inside a JSON object.

XML:

JSON:

JSON-P:

JSONP-X:

The way to invoke the JSONP-X output is to provide a format parameter of xml and a callback parameter.

This allows me for example to get the list of people I follow on twitter from my twitter homepage and display it in another document with a few lines of JavaScript without any need of using the API or having a local proxy:

More details on this are available in this blog post.

The annoying thing is that Placemaker only support POST request and does not have a JSON output - for now. GeoMaker allows non-developers to enter some text or a URL, filter the results (using YUI datatable) to remove false positives (no system is perfect) and get the embed code for a Yahoo Map or a list of microformatted locations as copy+paste. See the screencast to get the end user experience:

Of course, every time you build something like that, red-blooded developers will ask for an API to do the same thing (and pointing them to Placemaker wasn't enough). So here it is:

http://icant.co.uk/geomaker/api.php takes two parameters: url of the web document to load and output which could be map, kml, microformats, csv, or json (with callback for JSON-P). Using this you can analyze a url in JavaScript and get the data back as an array:

The Ajaxian.com output right now would be:

It includes compatibility tables, and will try to run the tests on your browser to give you feedback. It also includes sample code to check web browser support that you can use in your own projects.

• Introduction to the tests
• Demos of getter and setters in JavaScript
• JavaScript 1.8.1 tests
• Native JSON tests

Nice job Robert!

When you provide a JSON output for developers it does make sense to also allow for a callback parameter. That way your code can be used in script nodes without having to use any backend at all. Commonly this is called JSON-P and has been covered here in the long long ago. One of the issues with JSON-P is that when the callback method is not defined it throws an error.

The Bing API is the first instance where I have seen that they worked around that as the output is this:

I have no clue what the /* pageview_candidate */ is about and frown upon omitting the {} of the if statement, but I must say I do like this. One issue is that while end users don't get annoyed with errors, developers don't have a clue what happened either as the error is silent. One proposal would be to use a console.log() when there is an error:

More details about other ideas to improve this are here.

The latest beta of Persevere features a new native object storage engine called JavaScriptDB that provides high-end scalability and performance. Persevere now outperforms the common PHP and MySQL combination for accessing data via HTTP by about 40% and outperforms CouchDB by 249%. The new storage engine is designed and optimized specifically for persisting JavaScript and JSON data with dynamic object structures. It is also built for extreme scalability, with support for up to 9,000 petabytes of JSON/JS data in addition to any binary data.

This comparison isn't exactly apples-to-apples as it turns out–for the web app use case, Perservere has a bunch of value-adds on top of data storage:

Persevere/JavaScriptDB goes further [than relational DBs] with the flexibility to evolve schemas and handle partial schemas. Persevere also provides integrated server side JavaScript (SSJS) with persistence, Comet-driven data change notifications, JSONQuery, standards based HTTP interface with content negotiation, JSON-RPC interface to SSJS, cross-domain handling, CSRF protection, and more. All of these things are additional features that one would have to add to the stack for other storage systems, making them even slower. Persevere includes this functionality out of the box, while still maintaining extremely fast performance.

Kris spends a bit of time in his post explaining his test setup, but then gets to the good stuff:

So how does Persevere achieve this level of performance with the JavaScriptDB storage? The dynamic object-oriented nature of the data that is stored in JavaScriptDB is much different than that of a traditional relational database, so a number of innovative approaches were employed.

He goes into quite a bit of detail explaining the implementation details behind JavaScriptDB. The summary (with lightly edited quotes) is:

• Direct Data-Bound Object Representation: "In a traditional application stack, a record must have separate in-memory representations for [the database] and [the application] result set which then might be mapped to an object representation. With JavaScriptDB, the single in-memory object is reused for all result sets and data caching."
• Shared Cache of Objects with Copy-on-Write
• Append-based Database Storage: "Many traditional database commit data to a transaction log before committing data to the table, requiring multiple writes. JavaScriptDB appends transactional data directly to the main storage file; writes can be committed with a single IO operation."
• Adaptive On-Demand Concurrent Indexing
• Batched writes in integrity mode
• Pluggable Storage

Check out the full post all the details.

BrowserCouch is an attempt at an in-browser MapReduce implementation. It's written entirely in JavaScript and intended to work on all browsers, gracefully upgrading when support for better efficiency or feature set is detected.

Not coincidentally, this library is intended to mimic the functionality of CouchDB on the client-side, and may even support integration with CouchDB in the future.

Why?

This prototype is intended as a response to Vladimir Vuki?evi?'s blog post entitled HTML5 Web Storage and SQL. A CouchDB-like API seems like a nice solution to persistent storage on the Web because so many of its semantics are delegated out to the JavaScript language, which makes it potentially easy to standardize. Furthermore, the MapReduce paradigm also naturally takes advantage of multiple processor cores—something that is increasingly common in today's computing devices.

There is also an interactive CouchDB project that "is an emulator written in 100% JavaScript with tons of jQuery thrown in. It also implements the collation schemes as well as the map/reduce algorithms. While it doesn’t demonstrate replication, conflict management and a host of other capabilities in CouchDB, it does strive to illustrate concepts like schema-less JSON documents, map/reduce and how these things fit together."

You can check out the emulator here.

He has quickly reported back and is happy with the results.

Something like this:

will give you errors such as:

Invalid value for parameter 0. Property windowId: expected integer, got string.

It will be interesting to see how the APIs look when you go for this style throughout. Looking forward to see more.

The main findings of the team were that eval() is not only evil but also very slow whereas dynamic script nodes are fast but insecure. The solution was to do a custom evaluation of string data rather than using JSON:

Having set the performance bar pretty high with the last approach, we dove into custom data formats. The challenge would be to create a format that we could parse ourselves, using JavaScript’s String and RegExp methods, that would also match the speed of JSON executed natively. This would allow us to use Ajax again, but keep the data restricted to our domain.

Since we had already discovered that some methods of string manipulation didn’t perform well on large strings, we restricted ourselves to a method that we knew to be fast: split(). We used control characters to delimit each contact, and a different control character to delimit the fields within each contact. This allowed us to parse the string into contact objects with one split, then loop through that array and split again on each string.

Once this had been speeded up, all they needed to use was the YUI AutoComplete control and voilà - fast client side searches even with massive datasets.

JSONView is a new Firefox extension that gives you a nice way to view your JSON documents (JSONovich also does the trick).

Ben Hollis talks about the extension:

The extension itself is pretty simple. I wasn’t sure how to approach the problem of supporting a new content type for Firefox, so I followed the example of the wmlbrowser extension and implemented a custom nsIStreamConverter. What this means is that I created a new component that tells Firefox “I know how to translate documents of type application/json into HTML”. And that it does - parsing the JSON using the new native JSON support in Firefox 3 (for speed and security) and then constructing an HTML document that it passes along the chain. This seems to work pretty well, though there are some problems - some parts of Firefox forget the original type of the document and treat it as HTML, so “View Page Info” reports “text/html” instead of “application/json”, “Save as…” saves the generated HTML, Firebug sees the generated HTML, etc. Just recently I came across the nsIURLContentListener interface, which might offer a better way of implementing JSONView, but I’m honestly not sure - the Mozilla documentation is pretty sparse and it was hard enough to get as far as I did. I’m hoping some Mozilla gurus can give me some pointers now that it’s out in the open.

The native JSON API is part of the upcoming 3.1 revision of ECMAScript, so we should see it adopted in browsers pretty quickly. It’s also API compatible with json2.js, as you note, so many many web users will get the performance win without apps needing to update.

I suspect that the performance advantage for native JSON is even more pronounced on the encoding side, but I don’t have tests to hand to back it up.

This was a comment by Mike Shaver, VP of Engineering at Mozilla (disclaimer: where I work).

The comment related to the post on the native JSON support in Firefox 3.1:

In case you haven’t heard, one of Firefox 3.1’s awesome new features will be native JSON support. This is totally sweet for two reasons:

1. eval’ing JSON in the browser is unsafe. Using native JSON parsing protects you against possible code execution.
2. Safely eval’ing JSON with a 3rd party library can be orders of magnitude slower. Native JSON parsing is much faster.

How does native JSON work compared to plain old eval? Simple:

var jsonString = '{"name":"Ryan", "address":"Mountain View, CA"}';
var person = JSON.parse(jsonString);
// 'person' is now a JavaScript object with 2 properties; name and address

Pretty easy huh? And here’s how to get a JSON string from an object:

var personString = JSON.stringify(person);
// 'personString' now holds the string '{"name":"Ryan", "address":"Mountain View, CA"}'

“But wait!”, you say. “How is it safer? How much faster is it compared to eval?”. Ok, I’ll show you.

Gotta love it when browsers implement something that just starts to make apps faster without potentially knowing about it.

The purpose of JsonML is to provide a compact format for transporting XML-based data via JSON.

Simon Willison says it best:

An almost non-lossy serialization format for sending XML as JSON (plain text in between elements is ignored). Uses the (element-name, attribute-dictionary, list-of-children) tuple format, which sadly means many common cases end up taking more bytes than the original XML. Still an improvement on serializations that behave differently when a list of children has only one item in it.

Have you seen this path before? Hating on XML and going for JSON, but then trying to JSON as XML and at the end of it you just s/</{/g.

Just take a look at the GData JSON API