Saturday, September 2nd, 2006

Google Image Labeler: Collaborative Tagging Game

Category: Ajax, Comet, Games, Google, GWT, Usability

Google has released Google Image Labeler, a streaming Ajax app that makes it fun to label (tag) images apparently built with GWT (via TechCrunch). It’s a real-time collaborative app, where you work with an online partner, assigned by Google, to look at the same image and decide on some labels together. It works like this:

  • Google assigns you a random online partner.
  • You’re shown an image. Somewhere else in the world, your partner will see the same image.
  • You come up with as many labels as possible, until passing. Your partner is doing likewise.
  • When both of you come up with the same label, you both move on to the next image. You may also both pass.
  • Continue labelling images for 90 seconds. At the end, you’ll get a score based on how many labels you made. Scores are persistent if you’re signed in – there’s a table of high scorers.
  • Google now has tons of label data.

This is another example of streaming /Comet, in this case a novel application as well. However, there’s not a single, long-lived, connection. A little Firebug session shows a sequence of POST calls occurring, possibly one call for each label you make and one each time your partner makes a label. The calls are persistent in the sense that they stay open until something happens, but (in my case at least) there are many of them rather than a single call. Also, it looks like the entire interaction, including even the High Scores section, is delivered in a single client-side application, no page refresh.

Looks like this is built with Google Web Toolkit (Google’s first public GWT app?).

One surprising omission is keyboard shotcuts – right now, you can click Enter to submit a label, but you have to pick up your mouse and click to pass.

SearchEngineWatch dug out this excellent 51-minute presentation by Carnegie Mellon’s Luis von Ahn (also of CAPTCHA fame), who talks about ESP Game, which Google subsequently licensed to create Labeler. If you’re wondering whether the Labeller will help Google catch up on Flickr’s tags, here’s a salient stat from the video (8 minutes in): in one year, 9 billion people-hours are spent playing Solitaire; it took 20 million human-hours to build the entire Panama Canal – no wonder the professor talks about finding ways to optimise human cycles :-). Also, note that a single 90-second game will probably yield somewhere between 50 and 200 labels – admittedly some of them are rushed, but how long would it take to gather that info in most web apps? The professor speculates that Google could label all of its images in two months.

Posted by Michael Mahemoff at 4:53 am
18 Comments

++++-
4.2 rating from 90 votes

18 Comments »

Comments feed TrackBack URI

502 Server Error
Server Error
The server encountered a temporary error and could not complete your request.Please try again in 30 seconds.

Great fun, sadly one does get quite a few of the errors above, noticed by an XHR monitor. That and less experienced users ruin some of the fun.

Comment by Hans Duedal — September 2, 2006

Seems similar to the ESP Game:

http://www.espgame.org/

Comment by Paradise Pete — September 2, 2006

DUH the espgame.org IS the game made by this guy!

Comment by Mario — September 2, 2006

I think this is a great idea for the simple fact that they are trying to create a fun and interesting game to help them accomplish the goal of image tag relevance. People tend to forget that there is usually a purpose for many of these type of apps and give it a thumbs down saying ‘dumb game’. It is true, Google isnt the best at filtering adult material, and I had a couple run-ins with “a guest” while tagging, but overall this app shows what can be accomplished with some comet and/or GWT and strives to make one of their services better, for that I like it.

Comment by MindHacker — September 2, 2006

Trouble is they’re going to end up with a lot of lowest-common-denominator tags on the images. I’ve played this a few times and generally the matches I got with my partners were very generic so a picture of a man in a particular place or situation would end up tagged ‘man’ or a satellite photo of a particular place ends up tagged as ‘satellite’ with no indication of the actual place name.

The chances of two people brought together at random knowing more detail about a particular images is almost zero.

Comment by Doug Clinton — September 2, 2006

Wow! A fun and exciting way to work for nothing and help Google make billions at the same time! Where do I sign up?

Comment by Googo The Oppressor — September 2, 2006

Hi Doug, I found this too…conventions emerge quickly, like “man”, “two” (for two items), “old” for an old picture. But it seems – or Prof Ahn – have thought this through, as you’d expect. Here’s why it doesn’t matter so much:
– Labels become “off limits” as they’re established, meaning you can’t actually get a match from them, so effectively you can’t type them in. Over time, that ensures you’ll end up being more descriptive. It will make the game harder, but then there are always new photos, so there will be a variety of difficulty levels within the sequence. Also, the main “objective” is to get a high daily ranking, so even though overall scores will drop over time, the system already takes that into account by focusing on daily rankings.
– At first, I thought Google would only be interested in labels where two people match, but when you think about it, there’s actually heaps of data in all the unmatched labels you type as well. Maybe 100 people typed “man”, but meanwhile 20 typed “glasses”, 10 typed “beard”, 5 typed “john lennon”, and a few typed “beetles” … even if none of those actually got a match. Google could probably build a multi-choice game afterwards to see which of those actually make sense.

Comment by Michael Mahemoff — September 2, 2006

Hi Michael,

Ahh, I hadn’t appreicated that you can match against previously entered tags as well. Still, I’m not sure that a lot (any actually) of people are going to know that a particular satellite image is of the Nebraska mountains, and that image is pretty worthless without that knowledge. Still, I gues that outside the specialised areas like this it will be a great start on the task of labelling the images and a very interesting approach to doing it.

Thanks for a great podcast, btw, and I’ve found your Ajax patterns book really useful.

Comment by Doug Clinton — September 3, 2006

Doug, one other thing that’s not immediately apparent is personalization – any time an individual comes up with a label that a few others came up with (no more and no less than a few), there’s a good chance that guy knows something about that kind of image. By correlating against the well-established tags, you could establish some concept of specialization among the user base. e.g. If someone is good at geography, they’ll end up getting more geography images because they’ve produced rare terms for images labelled “map”.

Glad you like the podcast and book :-).

Comment by Michael Mahemoff — September 3, 2006

[…] da ajaxian Google has released Google Image Labeler, a streaming Ajax app that makes it fun to label (tag) images apparently built with GWT (via TechCrunch). It’s a real-time collaborative app, where you work with an online partner, assigned by Google, to look at the same image and decide on some labels together. […]

Pingback by CollaBlog » Blog Archive » Un “gioco” basato sul collaborative tagging — September 5, 2006

work for free! tag for Google’s Image Labeler!

as of last saturday, Google-fanboys all around the globe may help improving Googles image search results without getting paid seriously, Google released a new tool coined Google Image Labeler into public beta (sic!), inviting volunteers to add semant…

Trackback by nonsmokingarea.com — September 6, 2006

#2 woohoo

Comment by drdamour — September 6, 2006

Google sozialisiert sich weiter: Google Image Labeler

Google macht sich beim Thema “‘Qualität der Bildersuche” Gedanken: Google Image Labeler. Dass im semantischen Tagging viel Potenzial steckt, haben Dienste wie del.icio.us, Technorati & Co. bereits bewiesen.
Google setzt auf …

Trackback by PageTurner.info — September 8, 2006

[…] Google Image Labeler: Collaborative Tagging Game – A new tool from Google to make image searching easier. Mostly it’s interesting for the approach of paying people to do a job that’s easy for humans but hard for computers (similar to Amazon’s Mechanical Turk and Distributed Proofreaders). Third-Party Testing: Risky Business? – Looks at two types of security tests (vulernability and penetration) and risks of each. No Bug Database – How about fixing bugs when they’re found instead of entering them into a database to rot? Testing: Coverage Reports Considered Dangerous – John Casey’s bad experience when relying on code coverage statistics to tell him if his unit tests were adequately testing his refactoring project. Finding Great Developers – Joel’s strategies for finding and recruiting top developers. 30 minutes Flex test-drive for Java developers – Christophe Coenraets from Adobe put together a demonstration of what Flex can do. It’s as simple as deploying a .war file in your favoirte J2EE app server. The Usual Suspects – An entertaining list of different types of “architects” you’ll run into (e.g. the PowerPoint architect, the Gartner Architect, the UML Architect). Wanting to become an MCA : Step 1 – Self Assessment and Step 2 – Filling in the forms – Simon blogs about his initial experiences with the MS Certified Architect application process. When “Near Enough” Isn’t – “Too often, the adequate blocks the best when it comes to problem solving.” The SOA Magazine – A new online magazine all about SOA, published bi-monthly. […]

Pingback by Architect’s Linkblog » Blog Archive » 11 Links for 9/8/06 — September 8, 2006

Check out:

http://video.google.com/videoplay?docid=-8246463980976635143

Comment by Jonathan Bond-Caron — September 10, 2006

[…] Two days ago, my friend John sent me this Ajaxian article about Google Image Labeler. It’s a real-time collaborative game (using ajax), where you work with an online partner to look at the same image and decide on some labels/tags together. […]

Pingback by theory.isthereason » How Google’s using humans as “computers” — September 13, 2006

Aboslutely brilliant idea. I am trying to apply this to a video archive research project paper in one of my classes in Informatics here at the University at Buffalo, which I think will be a good way to solve national archive solutions. The “instant” nature of photos will be abstracted from this idea of contextual knowledge to film in a more or less 30-second bit of film and disecting more knowledge about a movie vs. the brief summary of events in a photo. More involved.. but the game knowledge fits longer more complex games. I’ll report back about this once it’s gets moving and give credit to the author of this approach.

Comment by John from Buffalo — September 13, 2006

[…] Ajaxian ” Games Top Ten Web 2.0 Games. Category: Fun, Games … 10 of some of the more popular Web 2.0 games offered on the web, as ranked by … […]

Pingback by Web 2.0 » Blog Archives » Recycled Content. Moved to Blosxom. JavaScript Native Interface — October 13, 2006

Leave a comment

You must be logged in to post a comment.