Natural Interaction

Our computer systems are hard to use. They enslave us rather than serve us. If we do nothing things will get worse, as billions of people and physical devices become interconnected. We need a radical change to a new breed of human-centered computers. We must simplify our computers in a big way.

That’s a great wish. But what do we really mean when we say we should “simplify”?

Our instinctual reaction is to equate simplicity with leanness – of features and of controls. That’s certainly a good avenue to explore. The Web has already shown the power of this approach; its single control – a mouse click on any highlighted phrase or image – has captured with its simplicity and ease of use hundreds of millions of enthusiasts. We should throw away 90 percent of the features and controls that come in today’s bloated software.

But cutting down the number of controls isn’t the full story, and can even lead to problems. For example, a typical digital watch has two buttons. One changes modes. Press it once to set the time, again to set the date, again to set the alarm, another time to set the day of the week. The other buttons lets you scroll to a specific time, date, or other information you want to set, whatever mode you may be using.

Even thought the watch has only two controls, this system is not simple, because it causes people to become confused and forget which mode they are in, and which procedure they should follow.

Maybe it’s not the number of controls that should be reduced, but the many different functions a system can perform. That sounds promising until you imagine a car that can only carry out two movements – go forward or not, and turn right or not. In principle, you can drive this car anywhere; if you want to turn left, you just keep turning right until you point to the left. But that ridiculous contraption is not simpler to use than your current car, which has many capabilities, like accelerate, brake, and so on. So minimizing the number of capabilities a system has isn’t really what we mean when we ask for simplification.

How about configuring a system with a control for every conceivable action we might want to take? That won’t do, either. Such a system would confuse every one of us, and result in unwieldy manuals and unwanted interactions among endless features.

Specialization can help simplification. The bottle opener and shoe-horn symbolize this time-honored tradition. In the Industrial Era we gave custom-made, motorized tools to doctors, mechanics, plumbers, and gardeners that simplified their jobs. But in the Information Age those same groups of people, plus musicians, accountants, engineers, and lawyers, are trudging along with the same word processor, spreadsheet, database, and graphics editor. Specialization has promise here for overarching ways to simplify computer systems, before specializing them through applications.

Maybe we are after an impossible task. I often hear the excuse that software systems are inherently so complex that they cannot be simplified. Nonsense! This is the mind-set of lazy engineers. Since time immemorial humans have strived to simplify the world so they can understand it. At a time when we are beginning to unravel incredibly incredibly complex natural systems like our genetic makeup, it’s outrageous to claim that the software we put together is too complicated to simplify.

Perhaps our call for simplification means we want a system whose machine actions match our human intent. When we use a VCR, our intent is to tape a specific show. Instead, we struggle to set the start and stop times of the recording, the recording speed, the channel number. Programming the machine in this way is so far from our level of interest and so frustrating that the experience has become the butt of endless jokes. We should be able to command “Tape tonight’s ball game” and be done with it.

Saying that a system should let us carry out our intent at our level and with little effort is tantamount to saying that it should be natural to use. You were not born with a keyboard or mouse socket in your belly button to communicate with the world around you. You were born with ears, a mouth, eyes, and the ability to listen, speak and see. Human-centric computing begs that you use these natural human capabilities to communicate with your machines so you won’t have to burden yourself learning new, complicated approaches to do what you already know how to do effortlessly.

Finally we have hit upon the biggest lever of system simplification: natural interaction with machines. It is the first step toward human-centric computing.

Here is an example of the kind of natural interaction we can expect by 2005. It is an exact replica of a spoken dialogue between human and machine, using a Year 2000 advanced prototype called “Mercury” at the MIT Lab for Computer Science.

It’s late Tuesday night and duty calls. You have to make a sudden business trip. You need to book plane flights right now. So you call the automated Mercury airline reservation system. It picks up, and a machine talks to you.

What is your name please?

Victor Zue.

Welcome Victor Zue. You have successfully logged onto Mercury. How may I help you?

I want to go from Boston to London on November 7.

Okay, from Boston to London on November 7. Can you provide the approximate departure time or airline preference?

I’d like to fly United.

Sorry, I’m not sure what you said.

I want to fly United.

Okay. United flights from Boston to London on November 7. One moment please …
I have a nonstop United flight leaving at 7:30 P.M. and arriving at 7:10 A.M. the following day.

Please book this flight.

United Flight 998 has been added to your itinerary. You depart Boston at 7:30 P.M. and arrive in London Heathrow at 7:10 A.M. the following day. Where would you like to go next?

I want to go to New York City two days later.

Okay. United flights from London Heathrow to New York City on November 10. One moment please ….. Can you provide a departure or arrival time? The earliest nonstop leaves at 9:00 A.M. and arrives at 12:15 P.M.

Please book this flight.

United Flight 957 has been added to your itinerary. You depart London Heathrow at 9:00 A.M. and arrive New York’s Kennedy at 12:15 P.M. Where would you like to go next?

I want to return to Boston that same evening.

Okay. United flights from New York’s Kennedy to Boston departing between 5:00 P.M. and midnight on November 10. One moment please …… I have three nonstop flights, a flight departing at 5:30 P.M., a flight departing at 7:25 P.M., and a flight departing at 9:10 P.M. Please select one.

Please book the second one.

United Flight 7282 has been added to your itinerary. You depart New York’s Kennedy at 7:25 P.M. and arrive in Boston at 8:47 P.M. Shall I price your itinerary?

Yes, please.

Pricing your itinerary will take a while. In the meantime, I’ll summarize your itinerary for you. On November 7, you depart on United Flight 998 from Boston at 7:30 P.M. and arrive in London Heathrow at 7:10 A.M. You arrive one day later. From there, on November 10, you depart on United Flight 957 at 9:00 A.M. and arrive in New York’s Kennedy at 12:15 P.M. From there, you depart on United Flight 7282 at 7:25 P.M. and arrive in Boston at 8:47 P.M. The price for your itinerary is $1,404.95. Shall I e-mail your itinerary to you?

Yes, please.

Your itinerary has been e-mailed to zue@xxx. Shall I start a new itinerary for you?

No, thank you.

Thank you for calling. Have a nice day.





Excerpt from “The Unfinished Revolution” by Michael Dertouzos