Tuesday, March 9th, 2010

Spectrum Visualization with the HTML5 Audio Data API

Category: Sound

The HTML5 specification introduces the

The above quote is from the Audio Data API extension that joins a bunch of juicy developer work in Firefox 3.7.

Thomas Sturm has taken that API and created a spectrum visualization a kin to some of the great work by Scott Schiller (using Flash).

There is a new onaudiowritten attribute:

  1. <audio src="song.ogg" controls="true"
  2.           onaudiowritten="audioWritten(event);">
  3. </audio>

that lets you get access to info such as the spectrum:

function audioWritten(event) {
spectrum = event.mozSpectrum;

var specSize = spectrum.length, magnitude;

// Clear the canvas before drawing spectrum
ctx.clearRect(0,0, canvas.width, canvas.height);

for ( var i = 0; i < specSize; i++ ) { magnitude = spectrum.item(i) * 4000; // multiply spectrum by a zoom value // Draw rectangle bars for each frequency bin ctx.fillRect(i * 4, canvas.height, 3, -magnitude); } } [/javascript] Add to that the ability to write audio.... [javascript] var audioOutput = new Audio(); audioOutput.mozSetup(2, 44100, 1); var samples = [0.242, 0.127, 0.0, -0.058, -0.242, ...]; audioOutput.mozAudioWrite(samples.length, samples); [/javascript] Nice work all around.

Posted by Dion Almaer at 6:20 am

4 rating from 27 votes


Comments feed TrackBack URI

So this technique relies on a proprietary attribute in Firefox?

Comment by smfoushee — March 9, 2010

smfoushee: They are making an experimental implementation and they hope that thier work will eventually be standardized in one form or another.

This is pretty cool stuff. This, couple with the File, drag & drop APIs makes me want to create a full featured audio player + visualization in Javascript

Comment by voidmind — March 9, 2010

Very neat to see progress on this front! The guys working on this experimental patch have been doing a great job (see this prior Ajaxian story for more background.)
If someone wants to experiment more with this stuff, I’d gladly support ripping apart my SoundManager 2 “360° UI” demo (screenshot) which shows waveform + frequency in a circle shape while the MP3 is playing. The UI is drawn via JS + canvas, the data comes from Flash – retrofitting it (or just reusing the canvas drawing bits) to use <audio> shouldn’t take much.

Comment by Schill — March 9, 2010

Leave a comment

You must be logged in to post a comment.