In our age of information and technology, there isn’t as much mystery as there used to be. In that sense, short URLs (e.g., tinyurl.com/123) can be fun! Who knows where you’ll wind up.
Some folks aren’t as happy with uncertainty in hyperlinking; one of them, Darragh Curran, wrote in to tell us about his project: Long URL Please.
Long URL please (http://www.longurlplease.com) is a JSON webservice to
efficiently convert short urls (tinyurl.com/123) to their originals.
I’ve got a simple jquery plugin to take advantage of it, and a firefox
plugin. It’s running on google app engine.
Darragh hates short URLs so much he’s offering to contribute his time to help wipe them off the face of the web:
I’d love to see it used in apps like twhirl/tweetdeck/twitterific, on
microblogging sites and pretty much anywhere that’s got lots of short
urls. In that respect I’ll happily contribute my time to help those
people/teams integrate with the service.
I like getting data from the web and I love JSON - as it is easy to use. The issue is that not many things on the web come as JSON from the get-go. Hence we need converters. You can use cURL and beautiful soup or roll your own hell of regular expressions. Alternatively you can use Yahoo Pipes to build your converter. Pipes is the bomb but a lot of people complained that there is no version control and that you need to use the very graphical interface to get to your data (which was the point of Pipes but let's not go there).
AlasRejoice for there is a solution now available and it is called YQL. YQL is a SQL-style language to get information from all kind of web services, and - using oAuth - even Yahoo's social graph. There is a test console available for you to get to grips with all the information it gives you access to (which is a lot!):
Here comes the kicker though: for all the open services that don't need authentication you can use these YQL statements as a REST API with JSON output and an optional callback function for JSON-P by adding it to http://query.yahooapis.com/v1/public/yql?. For example to get the latest three headlines from Ajaxian's RSS feed as JSON and wrap it in a function called leechajaxian do the following:
"abstract": "Introducing <b>JSON</b> <b>...</b> <b>JSON</b> (JavaScript Object Notation) is a lightweight data-interchange format. <b>...</b> <b>JSON</b> is a text format that is completely language <b>...</b>",
"title": "<b>JSON</b>",
"url": "http://www.json.org/"
},
{
"abstract": "The <b>JSON</b> format is specified in RFC 4627 by Douglas Crockford. <b>...</b> Although <b>JSON</b> was based on a subset of the JavaScript programming language <b>...</b>",
"title": "<b>JSON</b> - Wikipedia, the free encyclopedia",
"url": "http://en.wikipedia.org/wiki/JSON"
},
{
"abstract": "Matthew Morley has posted his PHP library for <b>JSON</b>-RPC 2.0. <b>...</b> I am think it is great to see the expanding <b>JSON</b> toolset available in JavaScript libraries. <b>...</b>",
"title": "<b>JSON</b>",
"url": "http://www.json.com/"
}
]
}
}
});
What about screenscraping? You can get data from any valid HTML document using XPATH with select * from html. For example to get the first 3 tag links on my blog you can do the following:
The team is working on making this easier - while we run every page that is indexed through tidy there is still a lot of choking going on (if people wrote valid HTML that wouldn't happen).
YQL is a pretty easy but also versatile language. You can even use complex aggregation and filtering by for example hosting a lot of URLs in a spreadsheet and loading them one by one before aggregating. The example given in the console is "select * from rss where url in (select title from atom where url="http://spreadsheets.google.com/feeds/list/pg_T0Mv3iBwIJoc82J1G8aQ/od6/public/basic") and description like "Wall Street" LIMIT 10 | unique(field="title")"
BOSS - Build Your Own Search Service (the your is silent for reasons I cannot tell you as it would endanger the lives of our agents in the field) is a Yahoo! API to access their search index and get the data back either as XML or JSON. Whilst there is ample documentation available it can still be a bit daunting to use the API purely in JavaScript - especially when you want to have several asynchronous calls - for example when you want to search the images, news and web sites for a certain query.
I've had several complaints of Hackers at the Open Hack Day in Brazil about this and wanted to make their lives easier by writing a wrapper for the API.
This wrapper is yboss and it was a big success at the Hack Day with the winning hack in the BOSS category actually being based on it.
To use the wrapper all you need to do is to embed it in your document
Then you have access to the get method of the wrapper which searches all the defined search options with the query you provided. You can define a callback method that will be called when all searches have been successfully performed.
The data is provided either as a JSON object with all the mandatory properties for a BOSS result display or - as shown here - as HTML lists that can be written out via innerHTML.
Ben Lisbakken, an ex-colleague from Google and all round good guy, has created a simple JSONP service (in the vein of json-time and html-whitelist) that calculates the users language based on browser headers:
Sam Ruby has that way about him that sees things very clearly. He just took a peak at jQuery for the first time and was able to really put into words what I think jQuery enthusiasts like about the library:
The notable thing about this is that despite all of the asynchronous events taking place, the code is sequential (nested, but sequential), and that the JSON results of the AJAX call is immediately available to the function that is invoked when the selection changes.
This is based on the following code that he wrote:
John Paulett wanted to be able to define complex Python model objects, then seamlessly pass them into CouchDB and to client-side Javascript.
To make this happen for objects that are beyond primitive sets he created JSON Pickle which has been used on the Universal Feed Parser, and lets you do the following:
Sometimes you can get in the zone just enough to be productive on a plane. On my flight to Mexico City yesterday, I created Endpoint a project that contains a server proxy, JavaScript client, and Greasemonkey Script with a mission. The mission is to take a URL, work out if it is a redirect (via a Location: header), and then return the final endpoint for it.
Why did I do this?
I was brainstorming functionality for a Twitter client with James Strachan (he is working on gtwit) and we talked about how annoying tinyurl / is.gd / snurl / you name it URLs are. They don't tell you where you are going, and you could get Rick Rolled (if you are lucky) or much much worse.
So, I wanted to create a library, and one client (Greasemonkey) to test it out. Then anyone else could use it too to resolve directly from their Web pages.
How does it work
You load up the JavaScript via script src and then you can call resolve, passing the URL and a callback that will get the result. A few examples:
Under the hood, a bunch of stuff is happening. I would love to be able to just use XMLHttpRequest to dynamically hit the URL and look at the headers, but the same-origin policy stops me.
This is why I have the server proxy, which returns a JSONP callback.
When you call resolve(url, callback) the script tag is created on the fly and added to the DOM. The callback function is all handled to allow multiple calls, and then the chain unravels.
Here you can see it all in action, showing how my Twitter stream will go through and the URLs will dynamically change from their tinyurl versions to whereyouaregoing.com:
, JavaScript
