Monday, September 20th, 2010
Video Conferencing with the HTML5 Device Element

Did you know that work is being done to enable videoconferencing from HTML5 applications? Ian Hickson has been doing work on the
The
Example usage:
- <p>To start chatting, select a video camera: <device type=media onchange="update(this.data)"></device></p>
- <video autoplay></video>
- <script>
- function update(stream) {
- document.getElementsByTagName('video')[0].src = stream.url;
- }
- </script>
The spec includes a way to work with Stream objects for the data coming from the device and to record Streams. It also includes an API for working with peer-to-peer connections, such as sendBitmap() or sendFile() to send data between a peer connection. The entire standard is still being baked but here is what a P2P connection might look like in pseudo-code:
- var serverConfig = ...; // configuration string obtained from server
- // contains details such as the IP address of a server that can speak some
- // protocol to help the client determine its public IP address, route packets
- // if necessary, etc.
- var local = new ConnectionPeer(serverConfig);
- local.getLocalConfiguration(function (configuration) {
- if (configuration != '') {
- ...; // send configuration to other peer using out-of-band mechanism
- } else {
- // we've exhausted our options; wait for connection
- }
- });
- function ... (configuration) {
- // called whenever we get configuration information out-of-band
- local.addRemoteConfiguration(configuration);
- }
- local.onconnect = function (event) {
- // we are connected!
- local.sendText('Hello');
- local.addStream(...); // send video
- local.onstream = function (event) {
- // receive video
- // (videoElement is some <video> element)
- if (local.remoteStreams.length > 0)
- videoElement.src = local.remoteStreams[0].url;
- };
- };





The aim is terrific, but I’m puzzled by the approach of merging two totally different things – selecting a data source, and streaming it elsewhere.
Instead of a new HTML element, wouldn’t it be better to use $gt;input type=”file” accept=”video/*”< to allow users to pick a webcam, and add a “StreamConnection” API for streaming any data (binary, JSON, etc)? After all, I bet web developers can think of a whole lot more to stream than just audio and video!
See it in action running in a custom webkit build
https://labs.ericsson.com/blog/beyond-html5-conversational-voice-and-video-implemented-webkit-gtk
I would LOVE it if they implement the RS232 device type. I’ve been forced to resort to all sorts of bad solutions to be able to bring serial data into a web browser. I use it for serial NMEA GPS recievers, and microcontroller projects where I’d like to input/output serial data to/from a web browser. Having a native way to read/write serial port data in a browser would be like a dream come true for me.
chrisfjay: The element really does return a File object. With windows, when you’re in a file dialog, you can enter a URL (in chrome, at least), and it’ll fetch that url, and still return it as a File object.
seems to be related to a Stream object.
Some differences are a finite fileSize, and possibly, bi-directional communication. There are some interesting developments in the XMLHttpRequest, BlobReader, FileWriter realm that certainly warrant reflection when developing a data device standard. XHR and Blob Reader both act on streams.
Hope this helps explain why is quite different than .
-Charles
Strip tags got stripped: “Hope this explains why [device] is quite different than [file]”.
There were some fun notions in the Plan 9 OS about treating file descriptors and streams as the same object type; and you certainly see bits of that in Linux/Unix; but overall, selecting a stream descriptor is very different than a file, in all modern OS.
HTML5 is really starting to look like it will be like the iPhone/iPad.
When people talk about cool stuff they usually say something like “there is an app for that !”
Now enter HTML5 “there is an API for that !” :-)
Is this an other nail in the flash/silverlight/whatever coffin ?
@leptons we’re in exactly the same boat, wet dream material to get hold of a barcode scanner in Javascript …