Touch is the new Click *

This blogger has moved!

I want to invite you to my new gig over at Light Thinking. We are devloping light gadgets for photographers. You can't be a designer and not be into photography at some level. Maybe you know about the whole off-camera strobe movement that originated over at at strobist? Maybe you even know about the new HDSLR cameras and all the amazing stuff people are doing with them? Visit Light Thinking

 

If you are a hardware person, touch is the new thin, if you are a software person then touch is the new click. If you are a little of both then I guess touch is the new thin click...

The need for feedback

Like many people, we have a car. Nothing fancy, 4 year old, reliable and not particularly high-tech. (Yes, the missus decided...) The car has an electronic flasher for the indicator lights/turn signals.

In the old days, cars used a thermal flasher. A thermal flasher is a mechanical device consisting of some pieces of spring metal and wire, cleverly arranged, inside a little tin can, to turn the blinker lights on and off every second or so. And it has a nice tick-tock-tick-tock sound.

The sound is important. The sound is the primary feedback mechanism for the indicator lights. You don't see the indicator lights from inside most cars. You could look down at the instrument panel, but you probably prefer to keep your eyes on the road, especially when driving through an intersection.

As time went by, electronic devices became less expensive than mechanical devices. But electronic devices do not generate sounds voluntarily. To solve this problem manufacturers had to fit a "sound circuit" and a "loudspeaker" into the electronic flasher to make it sound like an old-fashioned mechanical device.

Haptics

The current mobile phone touch screens are the equivalent of flashers without sound. Current touchscreen phones, like the iPhone, do not give adequate feedback. Look at the problems people have typing on the device. But all is not lost. Haptics is the "science of touch" and there is a lot of interesting work going on in that field.

Vibration as feedback

The vibration actuator common in most mobile phones is used for "announcements"; when you receive a call in silent mode, the phone vibrates. This rudimentary behavior can be expanded, and the idea would be to apply very detailed control over the vibration motor in order to give feedback from touch.

There are physical limitations to what you can do with vibration. The screen is fairly small and has to be made from a transparent and scratch resistant material. If the screen has a hard surface, there is currently no way to make only a part of the screen vibrate. If you apply mechanical vibration, the entire screen will vibrate. In many cases, this is OK, like when you have a single point of touch and you want to give a sense of "hit" or "miss".

I'm not at all sure that multi-touch gestures really need different feedback for each finger.

Problem with bandwidth

Experiments have shown that most people can distinguish between 6, 75 and 250 Hz vibration. They can also distinguish between sine and sawtooth vibration. This is not a lot of resolution. The problem with vibration is that the communication bandwidth is very narrow. It would be nice if is was possible to feel the keys on the control surface (the screen). To me, vibration seems inadequate for this type of "haptic rendering". It probably won't be possible to simulate texture and shape well enough. I think it would be possible to render any surface shape and size using vibration, but its hard for me to see how to create the sensation of an "edge". It would likely be a fuzzy object.

Audio + Vibration

A way to increase bandwidth and redundancy is obviously to combine vibration and sound. For example a little "tap" combined with one sound could mean a click. A little "buzz" combined with a different sound could mean an error.

Combining haptic and auditory feedback has a lot of promise and there is even a conference (there is always a conference!): 2nd International Workshop on Haptic and Audio Interaction Design. If you are interested in audio + vibration you should also take a look at this MIT Technology Review article and google Prof Stephen Brewster.

Movement matters

Imagine you touch something with your index finger. The contact area is about the size of a pea. If you hold your finger still, you get a somewhat imprecise impression of what you are touching. As soon as you move your finger you get a much more precise indication. If you hold still, you may not be able to tell if you touch, say, paper, plastic or wood. As soon as you move your finger over the surface, you can easily tell the difference. Humans are good at distinguishing texture and shape. Braille readers can distinguish 43 different shapes each with an 2,5 * 5mm (0.1 by 0.2 inches) area.

Instead of simulating via vibration, I want the real thing: adaptive surface texture and shape. Not only for the fingertip, but for the whole hand. There is relevant feedback to give also after the finger is lifted off the screen.

Liquid metal skin? Nanotech to the rescue!

We need a material that is transparent and durable in its resting state but can change its surface enough to make it possible for us to feel shapes. The material should also be touch sensitive. The entire phone should be covered in the stuff, no reason to limit the interaction to the screen only. The phone could be able to tell us when a download is completed or give directions to the post office while in the pocket or in the hand.
Btw, does anybody know what phone the T-1000 from Terminator 2 uses?

*) The title is snagged/derived from CEnriqueOrtiz. If you develop mobile phone software, I recommend you add his blog to your reader.