Monday, January 11th, 2010

Using YQL as a proxy for cross-domain Ajax

Category: JavaScript, jQuery, JSON, XmlHttpRequest, Yahoo!

OK, this is nothing shockingly new, but I found it pretty useful. Using jQuery, Ajax has become more or less a one-liner:

< view plain text >
  1. $(document).ready(function(){
  2.   $('.ajaxtrigger').click(function(){
  3.     $('#target').load($(this).attr('href'));
  4.     return false;
  5.   });
  6. });

This loads the document any link with a class of “ajaxtrigger” points to and updates the content of the element with the ID “target”. If the link is a third party link on another domain it fails though – and silently at that. Normally you’d work around that with a server-side proxy, but you can actually do without it.

YQL is a hosted web service that can scrape HTML for you. It also runs the HTML through HTML Tidy and caches it for you. For example to load you’d use the following statement:

  1. select * from html where url=''

As a URL this turns into:


You could request JSON-P by setting the output format to json and define a callback, but that would give the HTML back as a massive object which is not nice. YQL also offers JSON-P-X as an alternative which is a JSON object with a callback and the HTML as a string inside a simple Array. See it by clicking the following URL:*

Now, using jQuery’s getJSON() we can load this even without a named callback function. That way we can use one method for content that is third party and simply use load() for the other:

< view plain text >
  1. $(document).ready(function(){
  2.   var container = $('#target');
  3.   $('.ajaxtrigger').click(function(){
  4.     doAjax($(this).attr('href'));
  5.     return false;
  6.   });
  7.   function doAjax(url){
  8.     if(url.match('^http')){
  9.       $.getJSON(""+
  10.                 "q=select%20*%20from%20html%20where%20url%3D%22"+
  11.                 encodeURIComponent(url)+
  12.                 "%22&format=xml'&callback=?",
  13.         function(data){
  14.           if(data.results[0]){
  15.             container.html(data.results[0]);
  16.           } else {
  17.             var errormsg = '<p>Error: could not load the page.</p>';
  18.             container.html(errormsg);
  19.           }
  20.         }
  21.       );
  22.     } else {
  23.       $('#target').load(url);
  24.     }
  25.   }
  26. });

You can see the demo in action here, more details are available on my blog and the source is available on GitHub.

Posted by Chris Heilmann at 7:42 pm

3.6 rating from 37 votes


Comments feed TrackBack URI

I’ve been using YQL & JSON-P for a while on taking advantage of jQuery support for JSON-P with it’s getJSON(). But I personally prefer Prototype and stumble on Dan Dean’s Ajax.JSONRequest() – just in time for some Scripty2 fun!

Comment by Phunky — January 12, 2010

I *know* I’m in the minority on this, but I’ll say it anyway: I am NOT a fan of the jQuery chain syntax at all. As someone who develops enterprise software all day long, including maintaining said code (although thankfully I do little of that any more!) I really value code that is more readable over terse, quick-to-type code. To my eyes, that first bit of code would be horrible to have to decipher down the road (and yes, I understand it just fine right now… doesn’t mean my brain will accommodate me six months from now).

Like I said, I realize full-well that I’m in the minority on this, judging by the popularity of jQuery, so no need to flame. I can’t be utterly alone though, can I?!?

Comment by fzammetti — January 12, 2010

@fzammetti, chaining is brilliant, consider it a command line, where it goes wrong is passing anonymous functions, object and array literals directly as arguments.

var list = ["hello", "world"];
var message = list.join(" ");
// var to signify the function being thrown around.
var clickHandler = function(e) {
this.innerHTML = "Goodbye cruel world!";
return false;

Define vars before passing them to jQuery chained methods, chances are you will be able to reuse them as well on top of creating more maintainable code.

Comment by BenGerrissen — January 12, 2010

‘someHugeString’ should be ‘message’ ofcourse…

Comment by BenGerrissen — January 12, 2010

@fzammetti, You are definitely not alone, for me nothing is more visually appealing than Java or hardcore OO JavaScript! I don’t use any libraries or frameworks so this is no endorsement by any stretch of the imagination, but from purely a syntax perspective nothing looks better than ExtJS to me.

As for jQuery the reason I can’t stand its syntax extends beyond just method chaining, but also method overloading; using the same method as both a getter and setter (html, attr, css, etc.). Personally I would prefer an extra couple hundred lines of code to avoid that mess.

Comment by RyanMorr — January 12, 2010

I don’t quite get it… Aren’t essentially .load() and .getJSON() both just asynchronous GET requests? How come .load() fails and getJSON() gets through the same origin policy? Thanks.

Comment by pesho — January 12, 2010

pesho because load does an XHR request and getJSON loads JSON via a generated script node with JSON-P.

fzametti – thanks, no you are not alone. I have a tough time enjoying jQuery syntax, too.

Comment by Chris Heilmann — January 12, 2010

however if your robot.txt disallows crawling, YQL will not be able to pull anything back for you.

Comment by bizsimon — January 13, 2010

Leave a comment

You must be logged in to post a comment.