Monday, January 19th, 2009

BitTorrent in the Browser

Category: Browsers

Opera does it. Others talk about it. Robert Accettura posts about how he thinks we should implement BitTorrent in the browser.

He doesn’t want a X-Torrent HTTP header as we can use Accept-Encoding for the same goal. How would it work?

Assume my browser supports the BitTorrent protocol and I visit the following URL for a download:

http://dl.robert.accettura.com/pub/myfile.tar.gz

My request would look something like this:

Get: /pub/myfile.tar.gz
Host: dl.robert.accettura.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate,torrent
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://robert.accettura.com/download

The servers response would look something like this:

Date: Sun, 18 Jan 2009 00:25:54 GMT
Server: Apache
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: application/x-bittorrent

The content would be the actual torrent. The browser would handle as appropriate by opening a helper application or handling it internally. If I didn’t have torrent in my Accept-Encoding header, I would have been served via HTTP like we are all accustomed.

Now what happens if the server is not responding? A fallback to the DNS level could be done.

First take the GET and generate a SHA1 checksum for the GET, in my example that would be:

438296e855494825557824b691a09d06a86a21f1

Now to generate a DNS Query in the format [hash]._torrent.[server]:

438296e855494825557824b691a09d06a86a21f1._torrent.dl.robert.accettura.com

The response would look something like a Base64 encoded .torrent file broken up and served as TOR or TXT records. Should the string not fit in one record (I think the limit is 512 bytes) the response could be broken up into multiple records and concatenated by the client to reassemble.

Odds of a collision with existing DNS space is limited due to the use of a SHA1 hash and the _torrent subdomain. It coexists peacefully.

Downside

The downside here is that if your server fails your DNS is going to take an extra query from any client capable of doing this. There is slight latency in this process.

Upside/Conclusion

The upside is that DNS scaling has come a long way and is rarely an issue for popular sites and web hosts. DNS can (and often is) cached by ISP’s resulting in an automatic edge CDN thanks to ISP’s. ISP’s can also mitigate traffic on their networks by caching on their side (something I also suggested in 2004).

BitTorrent may be used for illegal content, but so is HTTP. I think costs for ISP’s and websites could be significantly cut by making BitTorrent more transparent as a data transfer protocol.

What do you think?

Posted by Dion Almaer at 7:12 am
8 Comments

+++--
3.7 rating from 15 votes

8 Comments »

Comments feed TrackBack URI

This is horrid. Not only is it an abuse of Accept-encoding (which indicates the values of “Content-encoding” response header which will be accepted), but it also goes on to suggest stuffing a file into DNS is a good idea.

Let’s reiterate:

– Uses the Accept-encoding header to signal a protocol switch.
– Suggests using the DNS to store files which could be megabytes in size and will only get bigger as time goes on.

Compatibility-wise I guess it doesn’t break anything, until lots of people start using it and ISPs find that 4gb is no longer sufficient RAM for their caching resolvers.

I couldn’t think of a worse solution to this problem.

Comment by dmw — January 19, 2009

I agree with dmw about the implementation, but the idea itself could be very, very useful.

Maybe if there was a better way to do it.

Comment by DoubleAW — January 19, 2009

i believe there are several firefox extensions that do this. right?

Comment by ajaxery — January 19, 2009

For metalink (FTP/HTTP/P2P failover & other features), we’ve been using transparent content negotiation which is apparently not the correct thing to do in some people’s opinion, but works. I’ve been told the HTTP Link header is the correct thing for us, so I’m hoping it’d be good for torrents too.

If anyone is interested in this, feel free to help us improve what we’re doing by dialoging and collaborating.

Comment by antini — January 19, 2009

EVERY! – not only Opera – browser should support the torrent protocol. The protocol itself has advanced over the years and with webseeds, we have something which can benefit from both worlds: direct downloads and “load balancing” by p2p.
.
If every browser would have .torrent support you could set up servers which acts like p2p clients and akamai would be out of business in no time. All server-torrent-client would deliver the downloads via the protocol and each downloading user would be give something back because he is uploading also.
.
Opensource websites like sourceforge.net which might have terabytes of download each month and can only be done by sponsoring are websites I would completly redo with p2p-torrent downloads if every browser would support it out of the box.
.
I think at least in the OS community there is a strong give-something-back mind, so if you use a software and you give upload back for your favorite software and save a developer costs of 10GB it (would) feel right.
.
So! call for action: Firefox, Webkit, IE .. support the torrent protocol please!

Comment by Aimos — January 19, 2009

@dmw: You didn’t read what I wrote….

1. Accept Encoding is correct since the server would serve a .torrent (over HTTP) as the response should it find the browser can accept it. The DNS layer is a fallback initiated by the browser should the HTTP download server fail. This is by far the most common point of failure in a high traffic environment. Accept Encoding doesn’t switch protocols as you claim it merely tells the server if it can offer a .torrent or not.

2. DNS would really hold only 1-2kb at most. It would hold the .torrent which generally run 1kb for essential info only. Megabytes is either a gross exaggeration or misunderstanding, I’ll assume the ladder. As I mentioned this is not really that different than storing SPF data in a TXT or SPF record. Downloading the actual file is up to the torrent network. ISP’s could cache the torrent itself if they wanted to keep it on their network and offer faster downloads to their customers.

Comment by raccettura — January 19, 2009

This is an interesting idea; I like the use of Content Negotiation with the .torrent file returned. I thought that we would have seen BitTorrent integrated into browsers and the web years ago, and I’ve been surprised that its taken so long. I know that Firefox has had an open patch for many years around this. Good for Opera taking a leadership role here.

Comment by Brad Neuberg — January 20, 2009

BitTorrent is a great technology, but it can be a bit hard to grasp for end users – especially that you must manually stop seeding the torrent.

Comment by kristianj — January 20, 2009

Leave a comment

You must be logged in to post a comment.