Monday, September 20th, 2010

Video Conferencing with the HTML5 Device Element

Category: HTML, Standards, Video

Did you know that work is being done to enable videoconferencing from HTML5 applications? Ian Hickson has been doing work on the element in a separate draft to make this possible.

The element will be used to allow the user to give permission to a page to use a device, such as a video camera. A type attribute can be used for the page to give more specificity on what kind of device they want access to. Right now this could be ‘media’ for access to an audio or visual device; ‘fs’ to access the file system, such as a USB connected media player; or ‘rs232’ or ‘usb’ to access devices connected in this manner.

Example usage:

  1. <p>To start chatting, select a video camera: <device type=media onchange="update("></device></p>
  2. <video autoplay></video>
  3. <script>
  4.  function update(stream) {
  5.    document.getElementsByTagName('video')[0].src = stream.url;
  6.  }
  7. </script>

The spec includes a way to work with Stream objects for the data coming from the device and to record Streams. It also includes an API for working with peer-to-peer connections, such as sendBitmap() or sendFile() to send data between a peer connection. The entire standard is still being baked but here is what a P2P connection might look like in pseudo-code:


  1. var serverConfig = ...; // configuration string obtained from server
  2. // contains details such as the IP address of a server that can speak some
  3. // protocol to help the client determine its public IP address, route packets
  4. // if necessary, etc.
  6. var local = new ConnectionPeer(serverConfig);
  7. local.getLocalConfiguration(function (configuration) {
  8.   if (configuration != '') {
  9.     ...; // send configuration to other peer using out-of-band mechanism
  10.   } else {
  11.     // we've exhausted our options; wait for connection
  12.   }
  13. });
  15. function ... (configuration) {
  16.   // called whenever we get configuration information out-of-band
  17.   local.addRemoteConfiguration(configuration);
  18. }
  20. local.onconnect = function (event) {
  21.   // we are connected!
  22.   local.sendText('Hello');
  23.   local.addStream(...); // send video
  24.   local.onstream = function (event) {
  25.     // receive video
  26.     // (videoElement is some <video> element)
  27.     if (local.remoteStreams.length > 0)
  28.       videoElement.src = local.remoteStreams[0].url;
  29.   };
  30. };

[via Aditya Mani]

Posted by Brad Neuberg at 6:00 am

3 rating from 8 votes


Comments feed TrackBack URI

The aim is terrific, but I’m puzzled by the approach of merging two totally different things – selecting a data source, and streaming it elsewhere.

Instead of a new HTML element, wouldn’t it be better to use $gt;input type=”file” accept=”video/*”< to allow users to pick a webcam, and add a “StreamConnection” API for streaming any data (binary, JSON, etc)? After all, I bet web developers can think of a whole lot more to stream than just audio and video!

Comment by chrisfjay — September 20, 2010

See it in action running in a custom webkit build

Comment by johnwards — September 20, 2010

I would LOVE it if they implement the RS232 device type. I’ve been forced to resort to all sorts of bad solutions to be able to bring serial data into a web browser. I use it for serial NMEA GPS recievers, and microcontroller projects where I’d like to input/output serial data to/from a web browser. Having a native way to read/write serial port data in a browser would be like a dream come true for me.

Comment by leptons — September 20, 2010

chrisfjay: The element really does return a File object. With windows, when you’re in a file dialog, you can enter a URL (in chrome, at least), and it’ll fetch that url, and still return it as a File object.

seems to be related to a Stream object.

Some differences are a finite fileSize, and possibly, bi-directional communication. There are some interesting developments in the XMLHttpRequest, BlobReader, FileWriter realm that certainly warrant reflection when developing a data device standard. XHR and Blob Reader both act on streams.

Hope this helps explain why is quite different than .


Comment by Downchuck — September 20, 2010

Strip tags got stripped: “Hope this explains why [device] is quite different than [file]”.

There were some fun notions in the Plan 9 OS about treating file descriptors and streams as the same object type; and you certainly see bits of that in Linux/Unix; but overall, selecting a stream descriptor is very different than a file, in all modern OS.

Comment by Downchuck — September 20, 2010

HTML5 is really starting to look like it will be like the iPhone/iPad.

When people talk about cool stuff they usually say something like “there is an app for that !”

Now enter HTML5 “there is an API for that !” :-)

Is this an other nail in the flash/silverlight/whatever coffin ?

Comment by SilentLennie — September 21, 2010

@leptons we’re in exactly the same boat, wet dream material to get hold of a barcode scanner in Javascript …

Comment by RowanM — September 22, 2010

Leave a comment

You must be logged in to post a comment.