Tuesday, September 15th, 2009

WAMI lets you add speech recognition to any web page

Category: Accessibility

<>p>Andrew Sutherland is giving a presentation at The Ajax Experience tomorrow at 9:30am where he will announce the availability of WAMI (Web-Accessible Multimodal Applications).

WAMI is a project out of MIT that lets you plug voice recognition directly into a javascript powered page, and optionally record+save audio files of people talking.

There are a couple of great demo videos that show it in action:

The “hello world” of WAMI is an application that listens to what you say and repeats it back to you, all via:

javascript
< view plain text >
  1. ar myWamiApp;
  2. function onLoad() {
  3.     //div in which to put the audio button, see below
  4.     var audioDiv = document.getElementById('AudioContainer');
  5.  
  6.     // JSGF grammar for your application
  7.     var jsgf =
  8.         "#JSGF V1.0;\n" +
  9.         "grammar parrot;\n" +
  10.         "public <top> = hello wami | i want a cracker | feed me;\n";
  11.  
  12.     var grammar = {"language" : "en-us", "grammar" : jsgf };
  13.  
  14.     //Audio options.
  15.     //pollForAudio: must be true for speech synthesis to work.
  16.     //If your application doesn't use speech synthesis, set this to false.
  17.     var audioOptions = {"pollForAudio" : true};
  18.  
  19.     //Configuration options.
  20.     //sendIncrementalResults: if true, you'll receive "incremental" recognition results as the user speaks
  21.     //sendAggregates: if true, you'll receive a "semantic" interpretation.  Your grammar
  22.     //must follow a specific format.
  23.     var configOptions = {"sendIncrementalResults" : false, "sendAggregates" : false};
  24.  
  25.     //Handlers are functions which are called for various events:
  26.     var handlers = {"onReady" : onWamiReady, //WAMI is loaded and ready
  27.                     "onRecognitionResult" : onWamiRecognitionResult,  //Speech recognition result available
  28.                     "onError" : onWamiError,  //An error occurred
  29.                     "onTimeout" : onWamiTimeout}; //WAMI timed out due to inactivity
  30.  
  31.     //Create your WAMI application with the settings and grammar we just created
  32.     myWamiApp = new WamiApp(audioDiv, handlers, "json", audioOptions, configOptions, grammar);
  33. }
  34.  
  35. function onWamiReady() {
  36.     //Called when your WAMI application is ready.  You might wait until now
  37.     //to tell the user it's time to start interacting
  38. }
  39.  
  40. //called when a speech recognition result is received
  41. //since we set sendIncrementalResults to false, this will only
  42. //be called after the user finishes speaking.  Otherwise,
  43. //it will be called many times as the user speaks.
  44. function onWamiRecognitionResult(result) {
  45.     var hyp = result.hyps[0].text;  //what we think the user said
  46.     alert("You said: '" + hyp + "'");
  47.     myWamiApp.speak(hyp); //Speech synthesis of what we heard
  48.     setTimeout("myWamiApp.replayLastRecording()", 500); //play back audio we recorded
  49. }
  50.  
  51. //called when an error occurs
  52. function onWamiError(type, message) {
  53.     alert("WAMI error: type  = " + type + ", message = " + message);   
  54. }
  55.  
  56. //called when your WAMI session times out due to
  57. //in activity.
  58. function onWamiTimeout() {
  59.     alert("WAMI timed out.  Hit reload to start over");
  60. }

Related Content:

7 Comments »

Comments feed TrackBack URI

For those trying and thinking that it froze, just wait… it’s Java so just wait…

Comment by emailandthings — September 15, 2009

Didn’t work at all for me. Does it expect an American accent?

Comment by Darkimmortal — September 15, 2009

Firefox complains because the applet has an expired cert.

Using Java or Flash just to access a standard input device feels so old. When will we get <input type=”audio”>?

Comment by randomrandom — September 15, 2009

I’ve stopped using Java (as both programmer & user) a loooong time ago.

I never surf with Java enabled. 99.9% of sites don’t need it. And now that Oracle is behind it, bleh…

-randomrandom2

Comment by randomrandom — September 16, 2009

As much as we try to kill Java applets they keep coming back because they can do things that strait up javascript can’t. As for the voice recognition it sounds like a neat concept. Wish I had a mic at work to try it ;-)

Comment by otatop — September 16, 2009

Leave a comment

You must be logged in to post a comment.