Monday, January 11th, 2010

Using YQL as a proxy for cross-domain Ajax

Category: JavaScript, jQuery, JSON, XmlHttpRequest, Yahoo!

<p>OK, this is nothing shockingly new, but I found it pretty useful. Using jQuery, Ajax has become more or less a one-liner:

javascript
< view plain text >
  1. $(document).ready(function(){
  2.   $('.ajaxtrigger').click(function(){
  3.     $('#target').load($(this).attr('href'));
  4.     return false;
  5.   });
  6. });

This loads the document any link with a class of “ajaxtrigger” points to and updates the content of the element with the ID “target”. If the link is a third party link on another domain it fails though – and silently at that. Normally you’d work around that with a server-side proxy, but you can actually do without it.

YQL is a hosted web service that can scrape HTML for you. It also runs the HTML through HTML Tidy and caches it for you. For example to load http://bbc.co.uk you’d use the following statement:

  1. select * from html where url='http://bbc.co.uk'

As a URL this turns into:

  1. http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D'http%3A%2F%2Fbbc.co.uk'%0A&format=xml

You could request JSON-P by setting the output format to json and define a callback, but that would give the HTML back as a massive object which is not nice. YQL also offers JSON-P-X as an alternative which is a JSON object with a callback and the HTML as a string inside a simple Array. See it by clicking the following URL:

http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%27http%3A%2F%2Fbbc.co.uk%27%0A&format=xml&diagnostics=false&callback=foo

Now, using jQuery’s getJSON() we can load this even without a named callback function. That way we can use one method for content that is third party and simply use load() for the other:

javascript
< view plain text >
  1. $(document).ready(function(){
  2.   var container = $('#target');
  3.   $('.ajaxtrigger').click(function(){
  4.     doAjax($(this).attr('href'));
  5.     return false;
  6.   });
  7.   function doAjax(url){
  8.     if(url.match('^http')){
  9.       $.getJSON("http://query.yahooapis.com/v1/public/yql?"+
  10.                 "q=select%20*%20from%20html%20where%20url%3D%22"+
  11.                 encodeURIComponent(url)+
  12.                 "%22&format=xml'&callback=?",
  13.         function(data){
  14.           if(data.results[0]){
  15.             container.html(data.results[0]);
  16.           } else {
  17.             var errormsg = '<p>Error: could not load the page.</p>';
  18.             container.html(errormsg);
  19.           }
  20.         }
  21.       );
  22.     } else {
  23.       $('#target').load(url);
  24.     }
  25.   }
  26. });

You can see the demo in action here, more details are available on my blog and the source is available on GitHub.

Related Content:

Posted by Chris Heilmann at 7:42 pm
8 Comments

+++--
3.6 rating from 35 votes

8 Comments »

Comments feed TrackBack URI

I’ve been using YQL & JSON-P for a while on http://phunky.co.uk/LastFM/ taking advantage of jQuery support for JSON-P with it’s getJSON(). But I personally prefer Prototype and stumble on Dan Dean’s Ajax.JSONRequest() – http://github.com/dandean/Ajax.JSONRequest just in time for some Scripty2 fun!

Comment by Phunky — January 12, 2010

I *know* I’m in the minority on this, but I’ll say it anyway: I am NOT a fan of the jQuery chain syntax at all. As someone who develops enterprise software all day long, including maintaining said code (although thankfully I do little of that any more!) I really value code that is more readable over terse, quick-to-type code. To my eyes, that first bit of code would be horrible to have to decipher down the road (and yes, I understand it just fine right now… doesn’t mean my brain will accommodate me six months from now).

Like I said, I realize full-well that I’m in the minority on this, judging by the popularity of jQuery, so no need to flame. I can’t be utterly alone though, can I?!?

Comment by fzammetti — January 12, 2010

@fzammetti, chaining is brilliant, consider it a command line, where it goes wrong is passing anonymous functions, object and array literals directly as arguments.

var list = ["hello", "world"];
var message = list.join(" ");
// var to signify the function being thrown around.
var clickHandler = function(e) {
this.innerHTML = "Goodbye cruel world!";
return false;
}
$(".app")
.html(someHugeString)
.click(clickHandler);

Define vars before passing them to jQuery chained methods, chances are you will be able to reuse them as well on top of creating more maintainable code.

Comment by BenGerrissen — January 12, 2010

‘someHugeString’ should be ‘message’ ofcourse…

Comment by BenGerrissen — January 12, 2010

@fzammetti, You are definitely not alone, for me nothing is more visually appealing than Java or hardcore OO JavaScript! I don’t use any libraries or frameworks so this is no endorsement by any stretch of the imagination, but from purely a syntax perspective nothing looks better than ExtJS to me.

As for jQuery the reason I can’t stand its syntax extends beyond just method chaining, but also method overloading; using the same method as both a getter and setter (html, attr, css, etc.). Personally I would prefer an extra couple hundred lines of code to avoid that mess.

Comment by RyanMorr — January 12, 2010

I don’t quite get it… Aren’t essentially .load() and .getJSON() both just asynchronous GET requests? How come .load() fails and getJSON() gets through the same origin policy? Thanks.

Comment by pesho — January 12, 2010

pesho because load does an XHR request and getJSON loads JSON via a generated script node with JSON-P.

fzametti – thanks, no you are not alone. I have a tough time enjoying jQuery syntax, too.

Comment by Chris Heilmann — January 12, 2010

however if your robot.txt disallows crawling, YQL will not be able to pull anything back for you.

Comment by bizsimon — January 13, 2010

Leave a comment

You must be logged in to post a comment.