Monday, January 19th, 2009
BitTorrent in the Browser
Opera does it. Others talk about it. Robert Accettura posts about how he thinks we should implement BitTorrent in the browser.
He doesn’t want a X-Torrent HTTP header as we can use Accept-Encoding for the same goal. How would it work?
Assume my browser supports the BitTorrent protocol and I visit the following URL for a download:
http://dl.robert.accettura.com/pub/myfile.tar.gzMy request would look something like this:
Get: /pub/myfile.tar.gz Host: dl.robert.accettura.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 Accept: */* Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate,torrent Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://robert.accettura.com/downloadThe servers response would look something like this:
Date: Sun, 18 Jan 2009 00:25:54 GMT Server: Apache Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Transfer-Encoding: chunked Content-Type: application/x-bittorrentThe content would be the actual torrent. The browser would handle as appropriate by opening a helper application or handling it internally. If I didn’t have
torrentin myAccept-Encodingheader, I would have been served via HTTP like we are all accustomed.Now what happens if the server is not responding? A fallback to the DNS level could be done.
First take the GET and generate a SHA1 checksum for the GET, in my example that would be:
438296e855494825557824b691a09d06a86a21f1Now to generate a DNS Query in the format
[hash]._torrent.[server]:438296e855494825557824b691a09d06a86a21f1._torrent.dl.robert.accettura.comThe response would look something like a Base64 encoded
.torrentfile broken up and served asTORorTXTrecords. Should the string not fit in one record (I think the limit is 512 bytes) the response could be broken up into multiple records and concatenated by the client to reassemble.Odds of a collision with existing DNS space is limited due to the use of a SHA1 hash and the
_torrentsubdomain. It coexists peacefully.Downside
The downside here is that if your server fails your DNS is going to take an extra query from any client capable of doing this. There is slight latency in this process.
Upside/Conclusion
The upside is that DNS scaling has come a long way and is rarely an issue for popular sites and web hosts. DNS can (and often is) cached by ISP’s resulting in an automatic edge CDN thanks to ISP’s. ISP’s can also mitigate traffic on their networks by caching on their side (something I also suggested in 2004).
BitTorrent may be used for illegal content, but so is HTTP. I think costs for ISP’s and websites could be significantly cut by making BitTorrent more transparent as a data transfer protocol.
What do you think?
This is horrid. Not only is it an abuse of Accept-encoding (which indicates the values of “Content-encoding” response header which will be accepted), but it also goes on to suggest stuffing a file into DNS is a good idea.
Let’s reiterate:
– Uses the Accept-encoding header to signal a protocol switch.
– Suggests using the DNS to store files which could be megabytes in size and will only get bigger as time goes on.
Compatibility-wise I guess it doesn’t break anything, until lots of people start using it and ISPs find that 4gb is no longer sufficient RAM for their caching resolvers.
I couldn’t think of a worse solution to this problem.
I agree with dmw about the implementation, but the idea itself could be very, very useful.
Maybe if there was a better way to do it.
i believe there are several firefox extensions that do this. right?
For metalink (FTP/HTTP/P2P failover & other features), we’ve been using transparent content negotiation which is apparently not the correct thing to do in some people’s opinion, but works. I’ve been told the HTTP Link header is the correct thing for us, so I’m hoping it’d be good for torrents too.
If anyone is interested in this, feel free to help us improve what we’re doing by dialoging and collaborating.
EVERY! – not only Opera – browser should support the torrent protocol. The protocol itself has advanced over the years and with webseeds, we have something which can benefit from both worlds: direct downloads and “load balancing” by p2p.
.
If every browser would have .torrent support you could set up servers which acts like p2p clients and akamai would be out of business in no time. All server-torrent-client would deliver the downloads via the protocol and each downloading user would be give something back because he is uploading also.
.
Opensource websites like sourceforge.net which might have terabytes of download each month and can only be done by sponsoring are websites I would completly redo with p2p-torrent downloads if every browser would support it out of the box.
.
I think at least in the OS community there is a strong give-something-back mind, so if you use a software and you give upload back for your favorite software and save a developer costs of 10GB it (would) feel right.
.
So! call for action: Firefox, Webkit, IE .. support the torrent protocol please!
@dmw: You didn’t read what I wrote….
1. Accept Encoding is correct since the server would serve a .torrent (over HTTP) as the response should it find the browser can accept it. The DNS layer is a fallback initiated by the browser should the HTTP download server fail. This is by far the most common point of failure in a high traffic environment. Accept Encoding doesn’t switch protocols as you claim it merely tells the server if it can offer a .torrent or not.
2. DNS would really hold only 1-2kb at most. It would hold the .torrent which generally run 1kb for essential info only. Megabytes is either a gross exaggeration or misunderstanding, I’ll assume the ladder. As I mentioned this is not really that different than storing SPF data in a TXT or SPF record. Downloading the actual file is up to the torrent network. ISP’s could cache the torrent itself if they wanted to keep it on their network and offer faster downloads to their customers.
This is an interesting idea; I like the use of Content Negotiation with the .torrent file returned. I thought that we would have seen BitTorrent integrated into browsers and the web years ago, and I’ve been surprised that its taken so long. I know that Firefox has had an open patch for many years around this. Good for Opera taking a leadership role here.
BitTorrent is a great technology, but it can be a bit hard to grasp for end users – especially that you must manually stop seeding the torrent.