Portfolio: MERL — Speech-based TV program search

Mitsubishi Electric Research Lab (MERL) is an R&D group for Mitsubishi Electric Corp. They developed a prototype allowing people to search TV program listings by saying the name of a program to find. (See the full technical paper (PDF, 2320 KB) for information on the underlying technology.) Before going further with their work, MERL staffers wanted to see how consumers reacted to it.

We learned important things to think about before making consumer product. See the full image below.

The focus was on getting the features in place; as with most early prototypes, visual design will come later.

See other work for MERL.

Results of the study

It was a fascinating study for a number of reasons:

People had their own ideas about how it should work. Participants demonstrated more usage scenarios than we predicted, showing how important early user research is. People seemed to think that as long as they were talking to the TV, they could ask for anything they wanted.

One person set up an unexpected scenario: "It's 7:00 and dinner is over. What's on at 8 so we can sit down and relax in front of the TV?" Others tried good ideas that were not supported in the prototype: restricting the search by favorite channels, or refining by speaking instead of clicking, neither of which was supported.

Recovery strategies. If the results didn't seem right, people tried interesting ways to recover. Examples: Changing their inflection, as if they were asking, rather than telling the system what to do ("Friends?"); highlighting part of the name ("Friendzzzzzzz") and repeating the request (which returned different results because of the underlying system). And. Saying. Individual. Words. Instead. Of. Continuous. Speech. Which. Never. Works. In. Speech. Recognition.

Users expected voice recognition to just work. The prototype returned all of the matching items, much like a text search online. That was too much because people wanted to see just what they asked for. This is a predictable result, but it clashed with some aspects the system design.

Participants didn't like scrolling through matching program names (see Interactions with the system, below), but looking through episodes for those programs was fine, maybe even entertaining. It was a more manageable list, and some people enjoyed reminiscing about favorite program episodes.

Interactions with the prototype

Thomson CompuMark UI example (new version)

The early prototype, with the three columns and program description area described below.

In the study, participants spoke the name of a program and saw these results:

A column of matching program names. If the first item wasn't right, they could scroll down. Examples"Earl" or "MLB Baseball"

A column with episode names . Once they found the right program, they could look for the episode they wanted to see. This isn't a very familiar concept to consumers. Examples: "Didn't Pay Taxes" for this week's episode of "Earl", or " New York Yankees at Boston Red Sox" for tonight's the baseball game or "Tampa Bay Devil Rays at Boston Red Sox" for tomorrow's.

FInally, select from the showing, which describe when the episode is on (including multiple shoings), the channel and related information.

The program description gave information about the story.

Back to previous page