Tuesday, March 9th, 2010

Spectrum Visualization with the HTML5 Audio Data API

Category: Sound

> The HTML5 specification introduces the

The above quote is from the Audio Data API extension that joins a bunch of juicy developer work in Firefox 3.7.

Thomas Sturm has taken that API and created a spectrum visualization a kin to some of the great work by Scott Schiller (using Flash).

There is a new onaudiowritten attribute:

  1. <audio src="song.ogg" controls="true"
  2.           onaudiowritten="audioWritten(event);">
  3. </audio>

that lets you get access to info such as the spectrum:

javascript
< view plain text >
  1. function audioWritten(event) {
  2.         spectrum = event.mozSpectrum;
  3.        
  4.         var specSize = spectrum.length, magnitude;
  5.        
  6.         // Clear the canvas before drawing spectrum
  7.         ctx.clearRect(0,0, canvas.width, canvas.height);
  8.        
  9.         for ( var i = 0; i < specSize; i++ ) {
  10.           magnitude = spectrum.item(i) * 4000; // multiply spectrum by a zoom value
  11.          
  12.           // Draw rectangle bars for each frequency bin
  13.           ctx.fillRect(i * 4, canvas.height, 3, -magnitude);
  14.         }
  15.       }

Add to that the ability to write audio….

javascript
< view plain text >
  1. var audioOutput = new Audio();
  2. audioOutput.mozSetup(2, 44100, 1);
  3.  
  4. var samples = [0.242, 0.127, 0.0, -0.058, -0.242, ...];
  5. audioOutput.mozAudioWrite(samples.length, samples);

Nice work all around.

Related Content:

3 Comments »

Comments feed TrackBack URI

So this technique relies on a proprietary attribute in Firefox?

Comment by smfoushee — March 9, 2010

smfoushee: They are making an experimental implementation and they hope that thier work will eventually be standardized in one form or another.

This is pretty cool stuff. This, couple with the File, drag & drop APIs makes me want to create a full featured audio player + visualization in Javascript

Comment by voidmind — March 9, 2010

Very neat to see progress on this front! The guys working on this experimental patch have been doing a great job (see this prior Ajaxian story for more background.)
 
If someone wants to experiment more with this stuff, I’d gladly support ripping apart my SoundManager 2 “360° UI” demo (screenshot) which shows waveform + frequency in a circle shape while the MP3 is playing. The UI is drawn via JS + canvas, the data comes from Flash – retrofitting it (or just reusing the canvas drawing bits) to use <audio> shouldn’t take much.

Comment by Schill — March 9, 2010

Leave a comment

You must be logged in to post a comment.