Computer mice, good at navigating cursors, and keyboards, helpful at entering data, are poor substitutes for sight, touch, and the sound of a voice. The ability to interact with computers as we do with people and other things has been evasive, but Microsoft and Livescribe, a startup, are inching closer.

Microsoft last week unveiled Surface, a tabletop computer running Windows Vista that responds to 52 points of touch, reads optical labels that identify physical objects, and recognizes Bluetooth and Wi-Fi devices. The touch technology in Surface allows people to "grab" items such as digital pictures on the screen, manipulate their size and orientation, and drag or flick them to a folder or across the screen.

Device recognition means that a wireless camera could automatically sync and download pictures to Surface, and a Bluetooth mobile device could upload them. Multitouch technology isn't new; Apple's forthcoming iPhone has it, too. But the fact that Surface recognizes so many touch points and different kinds of objects represents a meaningful advance.

Microsoft will release Surface at year's end to commercial partners, though consumers and most businesses will have to wait. Sheraton Hotels and Resorts will place Surface in hotel lobbies, where guests can use the systems to listen to music, send photos home, order food and drinks, and view maps. Hoyt Harper, senior VP of brand management for Sheraton, expects customers to catch on. "The beauty is that it's simple and literally hands-on," Harper says. Sheraton will start installing Surface at hotels in major cities later this year.

Microsoft CEO Steve Ballmer says "surface computing" could grow into a multibillion-dollar market, with future products having thinner designs, even functioning as monitors for "paintings" that hang on the wall. Microsoft envisions businesses using the technology to work collaboratively--marketing teams working together to design ad campaigns, for example. To make surface computing sell, hardware will have to get cheaper--early systems range from $5,000 to $10,000--and there will have to be more applications for it. That means Microsoft must get more partners involved in developing software and systems around the concept.

Microsoft's been expanding its portfolio of technologies with nontraditional user interfaces. The company last month closed its purchase of speech-recognition software company Tellme. "One of the big advances that we're investing literally billions of dollars in is the idea of natural interface," chairman Bill Gates said at a conference last month. "More and more, it is finally getting the kind of quality at the hardware and software level to move into the mainstream."

WHAT'S THE POINT?

Livescribe's Smartpen, due this fall, packs the processing power of a smartphone into the form of a pen, which is expected to cost about $200. Based on technology licensed from Sweden's Anoto, Smartpen will record up to 100 hours of audio, a speaker will play back audio and "speak" words and numbers, and a small organic light-emitting diode display on its side will show information.

A built-in camera in the tip of the pen takes 75 pictures a second to digitally record pen strokes. In the process, the pen records what it has written and can orient itself on special paper by using tiny ink dots to find its bearings. Smartpen can upload written notes to a PC or laptop without the need for a scanner.

Smartpen's most important initial application will be Paper Replay. Because audio and keystrokes will be marked by time stamps, and because the pen and paper incorporate a GPS-like system, users can tap the pen on a specific word or phrase in their notes and immediately hear back whatever was being said when the notes were being written.

Other potential applications for Smartpen: Users might write a math problem on paper and see the answer on the pen's display, organize and manage notes that have been synced with a PC or laptop, and look up written words in a dictionary, then view definitions on the pen's display. Smartpen will be able to translate a user's notes from script to text.

Businesses could use Smartpen for information management and retention, including protecting their intellectual property. By digitizing notes, for example, a pharmaceutical company might be able to track the first instance of a patentable invention. Interviewers and negotiators could record conversations by tapping the pen on paper while taking notes, then replay the recording for clarification, if necessary.

The technology sounds intriguing, but only if it works. Speech-recognition technology has been touted for years, but high error rates have hampered it from becoming a standard way of interacting with PCs and other computing devices.

Ease of use will be another test, for Livescribe and Microsoft's Surface computer. Bill Buxton, principal researcher at Microsoft Research, worked on multitouch technology as far back as 1984. "It's one thing to get the technology right," Buxton says. "It's another to get the key thing--the experience--correct."