Wednesday, January 21st, 2009

The new whitehouse is so open it wants Robots to come in too

Category: Editorial

Congrats to the new President, and to the great development team that created change.gov and the like, and now get to run whitehouse.gov.

Jason Kottke poked around and compared the robots.txt files from both administration.

The Bush version was ~2400 lines long:

User-agent: *
Disallow: /cgi-bin
Disallow: /search
Disallow: /query.html
Disallow: /omb/search
Disallow: /omb/query.html
Disallow: /expectmore/search
Disallow: /expectmore/query.html
Disallow: /results/search
Disallow: /results/query.html
Disallow: /earmarks/search
Disallow: /earmarks/query.html
Disallow: /help
Disallow: /360pics/text
Disallow: /911/911day/text
Disallow: /911/heroes/text
....

Obama?

User-agent: *
Disallow: /includes/

Nice to see the cleanup begin.

Also, if you view source on the new whitehouse.gov you will see many familiar libraries, including jQuery.

Posted by Dion Almaer at 12:18 am
10 Comments

+++--
3.1 rating from 32 votes

10 Comments »

Comments feed TrackBack URI

Haha Dion. You crack me up. :)

Comment by ibolmo — January 21, 2009

Are you sure it is the same development team? The White House site uses ASP.NET and the change site does not. From my experience developers don’t tend to jump between server-side technologies that much. Especially in the direction of ASP.NET.

Comment by JonBad — January 21, 2009

Unfortunately, saying they want to be accessible doesn’t earn them enough extra credit to pass 508 requirements.
.
http://www.whitehouse.gov/accessibility/
http://www.section508.gov/
.
Try tabbing through links, turning on high contrast, or using a screen reader to navigate the site. Site is pretty, yes, but not a very good user experience for those with disabilities, or even perfectly capable people like myself that like knowing where browser focus is.
.
For accessibility to be a part of the platform overall, this is disappointing.

Comment by phiggins — January 21, 2009

Wow. Ajaxian was the last site I expected there to be partisan politics in, hah. I highly doubt President Bush told the whitehouse.gov webmaster that he didn’t want Google crawling their site. It was probably just a really dumb webmaster.

One thing strange about the new whitehouse.gov is the terrible quality of the homepage jpg features. Other than that, it looks pretty nice.

Comment by fastspottim — January 21, 2009

whitehouse.gov Website Code Review here:
http://dotnetperls.com/Content/whitehouse-gov-Site.aspx

Comment by patrickm — January 21, 2009

The website got a “51” and lots of F’s from YSlow. It seems like Obama MUST CHANGE…

Comment by patitdude — January 21, 2009

I am getting really tired of seeing this posted all over the internet.
If you were to actually look through the old robots.txt it clearly shows that they were attempting to prevent the text only “print” versions of most of the pages from being crawled. This was likely an attempt to keep 2 versions of the same information in the search results and to make a better experience for visitors from search engines.

Comment by KenAuenson — January 22, 2009

Excellent point Ken. Also, “Bush” could have put in some honeypot urls in the robots.txt to track hack attempts. But since “Obama” doesn’t have any honeypot urls, then “Obama” must obviously be a better President for being more “transparent” to citizens and hackers alike. But hey, it’s just day 3.

Comment by bige — January 22, 2009

I’m little embarrassed but this is from the official site of Bulgarian government (witch, as it looks won’t be a government soon) – http://www.government.bg/robots.txt

Comment by RStankov — January 26, 2009

@Dion – so jquery has been part of whitehouse.gov since Obama? I thought it was very fresh the other day – more than just image changes. I saw it listed on css beauty. Obama seems real keen on making productive changes small & large :)

@patitdude – Obama’s changed site get fin yslow – but what did Bush’s get?

@RStankov – thats extremely funny

Comment by majic3 — January 27, 2009

Leave a comment

You must be logged in to post a comment.