For this assignment, you will experiment with and evaluate spoken dialog systems and report on your findings and experiences.

1. Evaluating current dialog systems:
   1. Call TellMe at 1-800-555-8355 and get two sets of driving directions between two different pairs of locations you know. Keep notes on your dialog with the system.
      For your homework report: evaluate the TellMe system with respect to recognition accuracy, speech output quality and dialog management. Issues of dialog management includes ease of navigation, responsiveness, error handling, on-demand help, plus support for short cuts and barge-in. You should also include any other information about your experience or impressions of the system that you feel is relevant.
   
   
   2. Call the following three different systems:
      • CMU's Roomline 1-877-CMU-PLAN or (412) 268-1084 http://www.cs.cmu.edu/~dbohus/RoomLine
        Meeting room booking system within SCS
      • CMU's Let's Go (412) 268-3526 http://www.cmuletsgo.org
        Port Authority bus schedules (primarily Oakland-based routes)
        Note: This system is publically accessible through the Port Authority in the evenings - you will not be able to get through to it 7-9 pm
      • AMTRAK's 1-800-USA-RAIL

      Use these systems for some appropriate task (i.e. schedule information, room availability). Describe your interaction with each of these systems, including (but not limited to) what you asked, if you got the intended information, any problems you encountered (and their nature). If it took multiple tries to successfully get information, include that in your report too.

      Compare these with each other and the TellMe system.

2. Building your own spoken dialog system:
   1. Register with TellMe Studio (http://studio.tellme.com) as a developer ("Join Studio"). During registration you will choose your own 5-9 digit developer ID. Include both of these in your report so that we may call your system.
   
   
   2. Write a simple application in VoiceXML. Your application, at the very least, should:
      • Greet the caller
      • Let the user ask a question about directors and movies
      • Give the answer
      At a minimum, your application should be able to answer questions about:
      • Star Wars (directed by George Lucas)
      • ET (directed by Steven Spielberg)
      • Blade Runner (directed by Ridley Scott)
      Hint: Use the TellMe developer documentation to find examples to modify. Start simply and add to it. Although the resulting dialog is somewhat trivial, the purpose of this homework is to allow you to understand some of the more practical aspects of building even a simple dialog system.


3. Reporting:
   Your report for this assignment should contain the following information:
   1. An evaluation section, including:
      • Your evaluation of the TellMe driving directions dialog system, as outlined above.
      • Your evaluation and experiences with the other three dialog systems, as outlined above.
      • A comparision of all four systems with each other.
   2. An implementation section, including:
      • Your 5-9 digit developer ID and PIN to access your system.
      • The structure of your application. It is strongly preferred that you use a diagram to describe the structure of your application (see http://studio.tellme.com/vxml2/ovw/applications-p.html for examples of such diagrams).
      • The descriptions of any documents, dialogs, subdialogs, grammars, scripts, event handlers, and variables (specifying scopes) used in your application, as well as the reasoning behind designing your system the way you did.
      • A sample (successful) dialog with your system and its structure. Optionally, you can also include a sample unsuccessful dialog to demonstrate what your application cannot handle and how it recovers from this kind of situation.
      • Commentary on whether your application can deal with problems such as no user input, wrong or inappropriate user input, appropriate but uncovered user input, etc., and if so, how it deals with these problems.
      • Commentary on how you could make your application deal with more elaborate questions, and more data.
      • Any other relevant observations you have from your experience building (and testing) your VoiceXML application.

You should submit both your report and the source code of your VoiceXML application. The contents of multiple VXML source files should be put into a single file when submitting via Blackboard. A significant percentage of your grade will be based on your writeup, both in terms of quality and detail, so you should take that into account.

If you have questions, check the HW4 FAQ or consult with either the TA or the instructor(s).