Interactive Voice Response (IVR) Case Study

Testing Your Telephone-Based e-Commerce Support

published in The Journal of Electronic Commerce, Volume 12, Number 2

<<Previous | 1 | 2 | 3 | 4 | 5 | Next>>

Your IVR Insurance Package

e-Commerce Depends on IVR
e-Commerce means commerce supported by electronic technology. This includes the ubiquitous telephone. How can we exploit its convenience, low-cost, and user familiarity for effective commercial applications?

Many organizations seek to reduce the burden of costly customer service staff by using telephone automation. For example, upon calling a credit card support number, the customer hears a recorded request, such as "Please enter your account number." The customer enters the digits using the telephone keypad. The recorded voice then offers options like, "To learn your current balance, press 1. To hear the last 10 transactions, press 2." and so forth. Last in the list of options, you hear "To speak to a customer representative, please press 0."

The bane of such IVR (Interactive Voice Response) systems occurs when the caller presses the "0" button. Because when the customer service representative (CSR) speaks with the caller, they start the "cost-meter." Each second carries a significant loaded labor rate. Some of our corporate customers have indicated that every second they save on the average length of a call means a saving of $120,000 a year. Where CSR employees number in the thousands, savings per second have been reported at a million dollars a year! (Yes, we were surprised, too.)

Given these incentives, organizations seek every opportunity to enhance the efficiency of their CSR and IVR systems. The CSR can work more efficiently when managers ensure their computer interface incorporates ergonomic or human factors principles of design (Schaffer, 1998). The IVR can work more efficiently – and keep customers from pressing the dreaded "0" button – by adoption of ergonomic design, as well. Note that the IVR is the first line of "expense defense." Failure of the IVR to assist the caller starts the clock for the more considerable CSR expense. Can we ensure against such failure?

Low-cost Insurance Premiums If you, a risk-averse manager, had a chance to purchase insurance that guaranteed the efficacy of your IVR, what would you pay? Would you spend a "person-week" on it? Or even two or three such weeks for a large and sophisticated system? Peanuts, you might reply, when thinking of the person years you have already committed. You invest in such insurance when you include "usability testing" in your IVR development cycle. Furthermore, the earlier you test, the less costly the insurance. This article will show you what’s involved and even empower your team to try it themselves. But first, let’s examine alternatives you may have already tried.

Alternatives to Early Testing Some IVR consultants make an interesting suggestion, based on the fact that frustrated callers will at some point quit the IVR protocol, call back, and then select the "rotary telephone" option given at the top of the your IVR menu. This option immediately connects the caller with the CSR because, presumably, the telephone lacks push-button tone technology. However, it has now become an expensive "by-pass" operation, neutralizing your IVR investment and accelerating your CSR costs. Obviously, you want to monitor this symptom. By using caller ID your IVR team can see who is dialing back. If they see such repeat calls, then they know your IVR has obstacles. If you have 800 or 900 access, your team can do similar analysis via the ANI.

An additional strategy would be to record data on each call. On the one hand, your team could audio-tape all calls (with warning to the caller that calls are monitored for quality control purposes). Your team would analyze the non-completed calls and find problems based on the sequence of IVR prompts, the tones of the entered options, and even uncomplimentary comments uttered by callers. On the other hand, your team could use software to log the number of times a given caller repeated a menu prompt without making a selection. Log other actions such as the menu item on which callers hang up, the length of pauses (i.e., confusing options), and repetitions from the same caller ID or ANI.

In all the above cases, note that you must have already constructed the IVR system to learn where the problems exist. In truth, this isn't insurance. And until you've made changes, it's pure risk exposure. Furthermore, system re-work costs far more than design testing.

Types of Testing If the idea of testing has captured your attention, let's discuss the degree of "fidelity" of such testing. Audiophiles and television home theatre consumers hotly debate issues of lo-fi and hi-fi equipment. Likewise, questions of low-fidelity testing vs. high-fidelity testing intrigues usability experts, as well. Several IVR researchers at GTE Laboratories, Inc. investigated the pros and cons of high-fidelity prototyping of an IVR system (Virzi, Sokolov, Karis, 1996). Their report provides instructive inspiration for low-fidelity testing and your consequent low-cost insurance program!

Their high-fidelity version of an IVR system included seven databases, over 500 sound files, and took two months to develop. It was an operational IVR system using an Apple Macintosh, but a mockup nonetheless. It used digitally recorded speech, and allowed caller identification, call screening, and automatic callback. The low-fidelity version used the "Wizard of Oz" technique in which one experimenter played the part of the computer, reading aloud what the computer would say. Subjects sat in the same room, and indicated which button they pressed in response to the statements. Ten college-age subjects performed the hi-fi tests, and another 10 did the lo-fi. All subjects received instruction to "think out loud" during the interaction. The sessions were videotaped.

<<Previous | 1 | 2 | 3 | 4 | 5 | Next>>