What is SpeakRight?

SpeakRight is an open-source Java framework for writing speech recognition applications in VoiceXML.Unlike most proprietary speech-app tools, SpeakRight is code-based. Applications are written in Java using SpeakRight's extensible classes. Java IDEs such as Eclipse provide great debugging, fast Java-aware editing, and refactoring. Dynamic generation of VoiceXML is done using the popular StringTemplate templating framework. Read more...

See Getting Started, Tutorial, and Table of Contents
Powered By Blogger

Thursday, June 14, 2007

Initialization

SpeakRight apps normally run in three different environments: in a JUnit test, in the interactive tester, and most importantly in a servlet. You can avoid problems by creating a single piece of initialization code that is used across all environments. This pace is called the app factory. It should be derived from SRFactory, which performs standard initialization.

Your class should override onCreateRunner and onInitRunner to do additional initialization, such as:
  • create and attach a model object
  • register prompt file(s)
  • set the extension point factory
  • other things. For example, the SimpsonsDemo app records votes in a text file, and its Voting object needs to be initialized with the path
Initialization is done using the createRunner method of SRFactory

public SRRunner createRunner(String projectDir, String returnUrl, String baseUrl, ISRServlet servlet);

The projectDir is a path to the application's base directory, which usually has sub-directories audio, grammar, and sro.

The two URLs are only needed in a servlet environment. returnUrl is the URL that the VoiceXML page should postback to. baseUrl is used to generate URLs for audio and grammars.

servlet can be null. It's an extension point that allows the servlet to do extra initialization.

Now let's look at each environment in turn.

JUnit

In a unit test, the dependencies can be visualized like this, from top to bottom:

JUnit test class
App (your callflow)
SRRunner
SRFactory or your derived class
SRConfig

Use your app factory to create a runner
AppFactory factory = new AppFactory();
SRRunner run = factory.createRunner();
Then run your application using the start and proceed methods of SRRunner.

If your app uses properties in the srf.properties file, you need to initialize SRConfig first. JUnit 4 has a per-class initializer called @BeforeClass

@BeforeClass static public void redirectStderr() {
SRConfig.init("C:\\source\\app2\\", "srf.properties");
}


Interactive tester

The interactive tester is a console app. It's The dependencies can be visualized like this, from top to bottom:

App (your callflow)
SRInteractiveTester
SRRunner
SRFactory or your derived class
SRConfig

SRInteractiveTester inits SRConfig for you.

SRInteractiveTester tester = new SRInteractiveTester();

AppFactory factory = new AppFactory();
SRRunner runner = factory.createRunner(appDir, "http://def.com", "", null);

App app = new App();
tester.init(app, run);
tester.run();

Servlet

In a servlet, the dependencies can be visualized like this, from top to bottom:

Servlet
App (your callflow)
SRRunner
SRServletRunner
SRFactory or your derived class
SRConfig

In a servlet the SRServletRunner class is used. You pass your app factory and it does initialization, including SRConfig. The SRRunner's project directory is set to the directory corresponding to the web apps' "/" url.

The code in doGet should be
SRServletRunner runner = new SRServletRunner(new AppFactory(), null, request, response, "GET");

if (runner.isNewSession()) {
SRRunner run = runner.createNewSRRunner(this);

IFlow flow = new App();
runner.startApp(flow);
}
else {
runner.continueApp();
}

The code in doPost should be
SRServletRunner runner = new SRServletRunner(new AppFactory(), null, request, response, "POST");

if (runner.isNewSession()) {
//err!!
runner.log("can't get new session in a POST!!");
}
else {
runner.continueApp();
}


SRConfig

SRConfig provides access to an srf.properties file. Properties are often used by the constructors of flow objects. Therefore it's important to initialize SRConfig early:

SRConfig.init(path, "srf.properties");

For console apps or JUnit, a hard-coded path is used. For servlets, this is done for you by SRServletRunner, which uses the directory corresponding to the web app's "/" base url.

Currently the SpeakRight framework itself does not use any properties, but applications are free to.

Thursday, May 24, 2007

List of Flow Objects

Flow objects are the building blocks of SpeakRight applications. Here is the list of available objects:
  • BranchFlow Performs branching in the callflow based on an application-defined condition
  • ChoiceFlow Branches based on user input, such as in a menu
  • DisconnectFlow Hangs up the call
  • FlowList a sequence of flow objects, optionally ending with an AppEvent
  • GotoUrlFlow Redirects to an external URL
  • LoopFlow Iterates over a sequence of sub-flows
  • NBestConfirmerFlow confirms NBest results
  • PromptFlow Plays one or more prompts
  • QuestionFlow Asks the user a question. Has built-in error retries for silence and nomatch.
  • RawContentFlow Ouput raw VoiceXML
  • RecordAudioFlow record the caller's voice to an audio file.
  • SRApp The root flow object
  • TransferFlow Transfer the call
  • YesNoConfirmerFlow used to confirm a single result
Additional flow objects can be created by implementing the IFlow interface.

There are also SROs (SpeakRight Reusable Objects) which you can use.

Call Control

The focus of SpeakRight is to be a framework for voice user interfaces. As such, it's support for call control is fairly limited. Consider using CCXML for applications with heavy use of call control. Or the RawContentFlow to generate platform-specific transfer and conferencing features.

SpeakRight catches the disconnect event (usually connection.disconnect) in order to do a final postback. This results in the DISCONNECT event being thrown and onDisconnect being invoked. SRApp provides a default handler for this.

There are a number of call control flow objects.

DisconnectFlow
This flow object plays a final prompt and hangs up.

SROTransferCall
This flow object transfers the call using one of the VoiceXML 2.1 transfer types: Blind, Bridged, or Consultation. The specified destination parameter is a string, such as
"tel:+2560001112222"
The format of the dial string is often platform specific.

TransferFlow has two prompts. An initial prompt called main is played before the transfer is initiated. A transferFailed prompt is played if the transfer fails to complete. Both these prompts can be overriden in an app-specific prompt XML file.

If the transfer fails because, for example, the destination is busy, then onTransferFailed is invoked. It plays the transferFailed prompt. Or you can override onTransferFailed and provide your own behaviour.

TransferFlow
This flow object is a low-level object. We recommend the use of SROTransferCall instead.

RawContentFlow
This flow object is an escape hatch. The application can supply any VoiceXML it likes. RawContentFlow may be useful for invoking platform-specific call control features.

Thursday, May 17, 2007

Optional Sub-Flow Objects

It's common in a speech application for some sections of the callflow to be optional. if the user is a preferred customer do X. Or if the app has forceLogin enabled then do Y.

Applications can simply create or not create certain flow objects based on the required logic. However, an alternative solution is provided called optional sub-flows. The BasicFlow and LoopFlow classes have been enhanced. The flow objects they contain (called sub-flows) can indicate that they don't wish to run by returning false from shouldExecute. When this occurs, the sub-flow is skipped and the next sub-flow is executed.

The advantage of optional sub-flows is that the decision on whether to run or not can be deferred until a sub-flow is executed. The initialization code doesn't need to handle this.

However there are a restrictions: the final sub-flow cannot be optional. Because if the final sub-flow returns null from its getFirst, there's nothing to execute.

Example code that creates the callflow using a BasicFlow object:

//callflow creation..
BasicFlow flow = new BasicFlow();
flow.add(new PromptFlow("Welcome"));
flow.add(new RegisterUser());
flow.add(new MainMenu());
app.add(flow);

And the class definition for the optional sub-flow looks like this:

class RegisterUser extends BaseSROQuestion {
public Model M;

@Override public boolean shouldExecute() {
return M.userHasRegistered;
}
//..rest of class definition omitted..

Note. I'm not completely happy with this feature. It's handy but the cant-be-last restriction will be easy to forget and will only fail at runtime. Full test coverage is required!



Wednesday, May 16, 2007

Supported Platforms

SpeakRight has been tested on these platforms:

  • Voxeo Evolution (free hosting site http://community.voxeo.com/). VoiceCenter 5.5
  • Voxeo Prophecy platform version 8.0 beta
The main mechanism for porting to a new framework is to customize the template file; see StringTemplate template engine.

VoiceXML Tags Supported

Here's an alphabetical list of VoiceXML tags supported by SpeakRight

  • assign name expr used to track dialog state
  • audio src play audio file
  • block
  • break time time is in msec
  • catch connection.disconnect So app gets a final postback
  • disconnect
  • exit
  • field name
  • filled
  • form one per page
  • goto next To goto an external URL
  • grammar type src type can be "text/gsl", "application/srgs+xml", or"application/srgs".
  • noinput count bargein main prompt(s), one per escalation
  • nomatch count bargein main prompt(s), one per escalation
  • prompt count bargein main prompt(s), one per escalation
  • submit next namelist method
  • transfer type dest connecttimeout
  • var name expr used to track dialog state
In addition, the RawContentFlow can be used by an app to output custom VoiceXML. It's useful for features not yet supported by SpeakRight.

You can customize the VoiceXML; see the StringTemplate template engine.

Thursday, May 10, 2007

Version 0.0.3 Released

Latest code drop is available at SourceForge (see Download link on the left). The project has been elevated to alpha status and can be used to build some real apps. The SimpsonsDemo app is an example of this, and is included as part of the release.