Spectrum Visualization with the HTML5 Audio Data API

Category: Sound

> The HTML5 specification introduces the

The above quote is from the Audio Data API extension that joins a bunch of juicy developer work in Firefox 3.7.

Thomas Sturm has taken that API and created a spectrum visualization a kin to some of the great work by Scott Schiller (using Flash).

There is a new onaudiowritten attribute:

PLAIN TEXT

HTML:

<audio src="song.ogg" controls="true"
onaudiowritten="audioWritten(event);">
</audio>

that lets you get access to info such as the spectrum:

PLAIN TEXT

JAVASCRIPT:

function audioWritten(event) {
spectrum = event.mozSpectrum;
var specSize = spectrum.length, magnitude;
// Clear the canvas before drawing spectrum
ctx.clearRect(0,0, canvas.width, canvas.height);
for ( var i = 0; i <specSize; i++ ) {
magnitude = spectrum.item(i) * 4000; // multiply spectrum by a zoom value
// Draw rectangle bars for each frequency bin
ctx.fillRect(i * 4, canvas.height, 3, -magnitude);
}
}

Add to that the ability to write audio....

PLAIN TEXT

JAVASCRIPT:

var audioOutput = new Audio();
audioOutput.mozSetup(2, 44100, 1);
var samples = [0.242, 0.127, 0.0, -0.058, -0.242, ...];
audioOutput.mozAudioWrite(samples.length, samples);

Nice work all around.

3 Comments »

Comments feed TrackBack URI

So this technique relies on a proprietary attribute in Firefox?

Comment by smfoushee — March 9, 2010

smfoushee: They are making an experimental implementation and they hope that thier work will eventually be standardized in one form or another.

This is pretty cool stuff. This, couple with the File, drag & drop APIs makes me want to create a full featured audio player + visualization in Javascript

Comment by voidmind — March 9, 2010

Very neat to see progress on this front! The guys working on this experimental patch have been doing a great job (see this prior Ajaxian story for more background.)

If someone wants to experiment more with this stuff, I’d gladly support ripping apart my SoundManager 2 “360° UI” demo (screenshot) which shows waveform + frequency in a circle shape while the MP3 is playing. The UI is drawn via JS + canvas, the data comes from Flash – retrofitting it (or just reusing the canvas drawing bits) to use <audio> shouldn’t take much.

Comment by Schill — March 9, 2010

Ajax Questions and Answers

Caching and dynamic views
My client does not want to change his cache settings on IE and I'm trying to figure out a way to dynamically update a view each time it is visited. I...

JQuery Function Not Defined?

Hi there,

I'm trying to set a php session variable from a radio button. The idea is that a jquery function is called which...

IE 8 Crashing Problem
Hello Everyone, Here\'s the situation: two iSeries boxes - Dev and Prod both V5R4M0 We\'re using AJAX to dynamically load content using a tabbed...