Tuesday, March 9th, 2010
Spectrum Visualization with the HTML5 Audio Data API
The HTML5 specification introduces the
The above quote is from the Audio Data API extension that joins a bunch of juicy developer work in Firefox 3.7.
Thomas Sturm has taken that API and created a spectrum visualization a kin to some of the great work by Scott Schiller (using Flash).
There is a new onaudiowritten attribute:
HTML:
<audio src="song.ogg" controls="true"
onaudiowritten="audioWritten(event);">
</audio>
onaudiowritten="audioWritten(event);">
</audio>
that lets you get access to info such as the spectrum:
JAVASCRIPT:
function audioWritten(event) {
spectrum = event.mozSpectrum;
var specSize = spectrum.length, magnitude;
// Clear the canvas before drawing spectrum
ctx.clearRect(0,0, canvas.width, canvas.height);
for ( var i = 0; i <specSize; i++ ) {
magnitude = spectrum.item(i) * 4000; // multiply spectrum by a zoom value
// Draw rectangle bars for each frequency bin
ctx.fillRect(i * 4, canvas.height, 3, -magnitude);
}
}
spectrum = event.mozSpectrum;
var specSize = spectrum.length, magnitude;
// Clear the canvas before drawing spectrum
ctx.clearRect(0,0, canvas.width, canvas.height);
for ( var i = 0; i <specSize; i++ ) {
magnitude = spectrum.item(i) * 4000; // multiply spectrum by a zoom value
// Draw rectangle bars for each frequency bin
ctx.fillRect(i * 4, canvas.height, 3, -magnitude);
}
}
Add to that the ability to write audio....
JAVASCRIPT:
var audioOutput = new Audio();
audioOutput.mozSetup(2, 44100, 1);
audioOutput.mozSetup(2, 44100, 1);
var samples = [0.242, 0.127, 0.0, -0.058, -0.242, ...];
audioOutput.mozAudioWrite(samples.length, samples);
Nice work all around.
So this technique relies on a proprietary attribute in Firefox?
smfoushee: They are making an experimental implementation and they hope that thier work will eventually be standardized in one form or another.
This is pretty cool stuff. This, couple with the File, drag & drop APIs makes me want to create a full featured audio player + visualization in Javascript
Very neat to see progress on this front! The guys working on this experimental patch have been doing a great job (see this prior Ajaxian story for more background.)
If someone wants to experiment more with this stuff, I’d gladly support ripping apart my SoundManager 2 “360° UI” demo (screenshot) which shows waveform + frequency in a circle shape while the MP3 is playing. The UI is drawn via JS + canvas, the data comes from Flash – retrofitting it (or just reusing the canvas drawing bits) to use <audio> shouldn’t take much.