Sender 11

Pecha Kutcha: Mobile Paradigms

This blogger has moved!

I want to invite you to my new gig over at Rift Labs. We are devloping Open Source Hardware for photographers. Come join us if you are into UX, design or photography at some level. Visit Rift Labs

 

A couple of weeks ago we had a Pecha Kutcha event at work. It was my first. If you don't know what Pecha Kutcha is, it's a presentation with 20 slides where each slide lasts 20 seconds. That makes your presentation 6 minutes 40 seconds long. It's about keeping the audience entertained.
I thought I'd talk about paradigms. Sounds daft, but it's really food for thought if you, well, take time to think about it. Which is precisely what you don't have in a Pecha Kutcha presentation...
I didn't have a lot of time to prepare, but I grabbed some previous slides from here and there and went for it. Here is the result, I've fleshed out my notes so they make sense to a reader.
If you are going to do a Pecha Kutcha presentation, don't do it this way, ha-ha. Way too much content. It's about 1/3 too much text, but a larger problem is that I'm trying to touch on too many points. You really need to limit the number of points you want to touch on, because the audience will need more than 20 second to digest each.

I didn't put in all the slides here, but I've numbered each 20 second slot as you see below.

1. We are in the business of organizing stuff visually, - not for ourselves, but for other people.
   One major problem we, as Interaction Designers, have is to understand how those “other people” think. We can't read other peoples minds just yet. Since people talk more than they draw, we listen to how they talk. What they say isn't that interesting, really. Lets be honest, he-he. It's how they talk, what words they use, that is interesting.
    
2. When we design something, we have a mental model in our head of how the thing work. And we use metaphors to make our design understandable. The metaphor defines the personality of the product. Using the product is basically a dialogue between the designer and the end-user, just separated in time. You put a button “here” telling the user “click me”. After the user clicks you say something else - it’s a dialogue.
    
3. How well does the metaphors we choose fit the users conceptual model? What do the people you design for bring to the problem you are trying to solve? A lot of their existing thinking is going to be a part of the system you end up designing - whether you are aware of it or not!
   If they look at something and think “Oh, it’s a thingamabob”, when it's really a doohickey, you’re in trouble.
    
4. And we are witnessing a pretty serious paradigm shift right now. The phones themselves are changing, the OS’es are changing and the form factors are changing.
   We invite users to enter a place or use a piece of software that we have created. In order to make it understandable and coherent, we like to use metaphors. The desktop metaphor is well known from the computer, we have used it for 25 years now. "Files" are in "Folders", the “Trashcan” is for deleting stuff and everything is placed on a "Desktop". The Desktop metaphor is the dominant organizing principle for the computer. Bill Verplank talks about Interaction design paradigms, or "computer paradigms":
    
    
5. Computer as Brain
   An early metaphor was the Computer as a Brain. Thinking machines. The machines will take over thinking for us.
   If the computer is a brain, then how intelligent is it? How do I talk to it? Can I use spoken language, handwriting? Artificial intelligence. You can see where this leads to all sorts of efforts to develop artificial intelligence where the goal often is a computer that  know how to do things for us.
    
6. Computer as Person.
   The computer is an agent, an avatar. The computer becomes a “personal assistant”. It pops on screen, is friendly, talks to you. It also distracts you. Give the computer some high-level instruction and it scoots off and come back with what you want.
   Verplank: An agent-based interface is going to require a lot of trust. And so far no one has been able to demonstrate anything remotely worthy.
   By the way, I think The Semantic Web (Web 3.0) is really a revisit of "Computer as Agent".
   
    
7. Computer as life is an extension of Computer as Person. You could argue that computers are already a life-form of sorts. Computer viruses are here and they can multiply. The robots are coming. Let's hope they will be vegetarian, not carnivores.
   Verplank: Should systems have magic in them? No, computers should not have magic.
   
     
8. Computer as Tool
   This paradigm is all about efficiency, empowerment etc. Users have tasks, they have jobs to do. The computer provides tools that empowers them to do good jobs. It's all very IBM and "Microsoft". This tool paradigm is where "usability" comes from. A good tool is half the work, right? Computer as Tool uses a desktop metaphor.
    
9. When computers became a tool, we started doing direct manipulation, grabbing something and using it directly.
   Verplank: Interaction style changed radically. Before this we "set up" a task; We entered data, adjusted levers and then clicked "Go". The computer went to work and after a while spat out the result.
   Web forms are indirect manipulation. Enter a bunch of stuff and click "Submit". AJAX promises to bring "toolness" back to the web.
   
   
10. Computer as medium
    The Computer as Medium: Right out of Nicholas Negroponte's Media Lab.
    Multimedia. Ways to publish, communicate, persuade, engage and entertain. Convey a message and deliver content. Doesn't this sound a lot like the media industry? Computer as Medium use a page metaphor from the web.
    Lately we’ve seen a lot of the river metaphors like in Facebook, Twitter, and a lot of others. Strong sense of time in this paradigm. 
     
    
11. What are the Mobile Phone paradigms?
    Ever since the phone manufacturers started adding features they have tried to figure out what they are actually making.
    What are people calling their phones? What are we as designers calling the phones? What is marketing saying? We're not using the same words.
     
12. A phone is a phone
    For many people, for most people actually, the phone is semitransparent – it has a very sub-servant personality. When people pick up the phone to call someone, they don't really see the phone. Their inner eye see an image of the person they are about to call or send a message. So they dont feel like they are interacting with the phone much.
    
     
13. Phone as computer
    “Smartphone” is a marketing term, “dataphone” is a people term.
    Originally Apple didn’t want the iPhone to be a computer. But developers saw it as a computing platform and started hacking it. But it didn't become a computer to the users. Still, smartphone-users interact directly with the phone. It is no longer semitransparent.
     
14. Phone as media
    Is the iPhone designed as an entertainment device? No, but it has become one. If you look at what sells in the App Stores, host people see them as entertainment devices.
     
15. Converged devices
    Nokia: Multimedia Computer. This is the voice of convergence delivering a (weak?) marketing message. Or the message is good, but the execution is not.
    Sony Ericsson has 3 product lines: Walkman Phones, Cybershot Phones and Smartphones. They have tried to brand these segments separately with good success. Now in trouble since the feature phone segment collapsed and their smartphone OS went the way of the Dodo.
     
     
16. Phone as Fashion
    Most of us don’t consider ourselves to be in the fashion industry. Apple is clearly into fashion when it comes to their products. According to Steve Jobs you have to buy a new iPod once a year in order to keep up. The iPhone would probably follow the same pattern was it not for the 18 or 24 month contracts.
    Apple markets a bolt of lightning that connects to your ears and lets you dance down the street.
     
17. If the phone has become a computer, what kind of computer is it?  A brain? Media? A Tool? Computer as person. Maybe it is neither.
     
18. Phone as a container
    Phone as container for your "life story". Christian Lindholm of Nokia has worked a lot on this with "Lifestreaming". The place that stores the content of my life. But is it on the phone or is it really on the internet, - and the phone is just a way to access it? The answer to that question is important for anyone who designs software and services for mobile.
    
    
19. The phone is my room, my personal place, my purse. What I have there is not for others to see? I share with others that I wish to communicate. Family, Friends. Flirting. It also have a work connection there, after all work is part of my life. Work and personal may exist on the same phone, but are separate parts of my life. I think this is what we are currently thinking as designers, when we work on mobile phone software. So let's hope the users see it the same way :-)

Somewhere in there, there was one more slide...but I can't remember what that was. Anyway, I encourage you to check out Bill Verplank for more on computer paradigms.

The Multi-Field Editor

This blogger has moved!

I want to invite you to my new gig over at Rift Labs. We are devloping Open Source Hardware for photographers. Come join us if you are into UX, design or photography at some level. Visit Rift Labs

 

My earlier post about the mobile Text Input Field was met with quite a bit of interest. A little surprising perhaps, since it's a somewhat mundane problem area. But understandable, because its so important to get right. So, to complement it, here is an input control for bound numerical data. 

The little brother of the text input field is the Multi-field editor. While the Text input field is suitable for general free form text and numbers, it is not ideal for structured data like dates, time, monetary values, number ranges, etc. 

Multi-field editors are numeric editors with multiple linked and bound fields. To the user it apperas as a single control with the selector moving from value to value. An editor for entering time would typically have two fields, for a date three fields and for entering an IP-address, four fields.

Most mobile OS'es have Multi-field editors and they behave somewhat differently. Sony Ericsson uses a mix of numbers and text (not so great), Microsoft selects each digit separately, Nokia selects values instead of digits. Apple (touchscreen) has big slot machine-type date and time pickers, but they are Multi-field editors all the same.

Example

To get a feel for how it works, you should play around with different phones, a text explanation is not enough. But let's look at an editor for a Date:
A Multi-field editor for entering a date consists of 3 input fields, one for the day, one for the month and one for the year. The fields are linked; the selection moves from field to field. A Multi-field editor is never empty. It always displays a value, for example today's date.

A number-key press replaces the existing number with a new. The next number-key press is interpreted as a second digit. Since a maximum of 2 digits is possible for a day number value, selection is now moved to the next field. If the user wants to enter only a single digit value, they can move to the next field with the arrow keys (or precede the digit with a zero).

If the number entered is higher or lower than allowed, the user is alerted. Use visual, audible and haptic alerts, or a combination thereof.
The 3 input fields are interdependently bound. The highest allowed day value depends on the value in the month and the year fields.

Data types

These are typical data types for Multi-field editors :

• Number 
• Date 
• Time
• IP-number
• Number range

The individual fields of a Multi-field editor typically have these properties:

• Maximum value 
• Minimum value 
• Maximum length (the number of digits) 
• Default value (there is always a starting value)
• Delimiter (often depends on locale)

The Date Multi-field Editor typically has these properties:

• Minimum Date
• Maximum Date
• Date format:
• dd.MM.yyyy
• MM.dd.yyyy
• yyyy.MM.dd

The Time Multi-field editor typically has these properties: 

• Minimum Time
• Maximum Time
• Time format:
• hh:mm
• hh:mm:ss (in case you really, really need the seconds)
• hh:mm AM/PM (since AM/PM is a binary selection, you can toggle on any key-press)
• hh:mm:ss AM/PM

etc.

Multi-field editors in Nokia Series 40 Edition 5.

Active validation

The Multi-field editor actively validates input. If you try to enter '40' into a date field you are alerted as you type. By the time you have entered a complete date, a time, an ip-number etc, the system knows that the number is valid. There is no need for a roundtrip to the server just to check the format.

But what about calendars?

I hear you saying: Ok, fine for time or monetary values, but a calendar surely must be better for entering a date?

A Multi-Field Editor is smaller and takes up less screen real-estate. You can edit in-place and see the rest of the screen. Usually it is easier to type in the date than to pick it from a calendar, especially if you need to enter the year. For example if you need to type in a date of birth.

A calendar always takes over the entire screen. A calendar is great if you want to show the date in context with other dates, for example if you want to schedule a meeting and want to find a suitable time. Each has its strengths depending on what you need it for.

But whatever you do, don't use dropdown lists for entering dates! :-)

Mobile Design Showcases

This blogger has moved!

I want to invite you to my new gig over at Rift Labs. We are devloping Open Source Hardware for photographers. Come join us if you are into UX, design or photography at some level. Visit Rift Labs

 

 

Nokia has just published a document with 10 examples of what they consider good mobile design. It is a mix of applications and mobile web sites. Many of the usual suspects, but also a couple of lesser knowns. 

I do not necessarily agree with everything. (Surprise, right? :-)) One of the criteria is "Visual WOW" and Jaiku Mobile is considered "stylish". To me Jaiku Mobile looks like its designed by a MySpace user high on banana peel and bad Klingon poetry. The application runs on top of whatever Theme the user has chosen for their phone, so maybe unfair to criticize it. Anyway, "lively" was probably a design objective. Jaiku was acquired by Google last year and they closed new sign-up. So unless you know an existing Jaiku user, you won't be able to check it out. 

Fring is another of the examples. I tried it quite a while ago, but I just couldn't figure it out. Setup required me to configure my "IP telephony signaling protocol", so I backed out, thinking "maybe next version". Fring makes calls and messaging over WiFi and is a way cool concept. Maybe it's time to give Fring another try. 

Good examples and interesting evaluations. Definitely worth having a look, as the examples are some of the most used 3rd party mobile applications out there today. You can find the document here.

The anatomy of a text input field

This blogger has moved!

I want to invite you to my new gig over at Rift Labs. We are devloping Open Source Hardware for photographers. Come join us if you are into UX, design or photography at some level. Visit Rift Labs

 

There are two interface interface elements of typical mobile user interfaces that does the heavy lifting. That is the Text Input Field and the List. I thought I'd share with you some of the usability considerations that is going into Text Input Fields at mBricks.

The keypad is well suited to number entry, but ill suited to text entry. Entering text on a mobile phone is hard, and people tend to avoid it if they can. The less text input you have, the better. But you can't avoid it all together.

Text input fields are suitable for plain text, numbers and passwords. For entering specific data types like date, time, number ranges etc. you should not use a text input field. Instead use Multi-field Input or a selection widget like a Calendar.

Elements of the Text Input Field

Full screen editing

Full screen edit

Edit-in-place is preferable in most cases. A person can see the context, and the visuals is consistent. Full screen editing is preferable when the user is likely to enter a large amount of text. Predictive text (T9, etc) may only be available in full screen edit.

All phone OS'es offers a full screen editor. Use a visual clue like the one above (to the left) to indicate that the full screen editor will open.

Vertical and horizontal layout

Most input fields consists of the control itself and its label. Screen layout may be horizontal or vertical.

In a vertical layout the label is positioned above the field, and both the label and the input field are left aligned. In a horizontal layout the label is right justified and the input field is left justified.

Input prompt

Instead of a label, you can use a prompt displayed inside the field itself. The prompt takes less screen real estate, but is only visible when no text is entered.

Error prevention: constraining input

When the input is to be consumed by a human being, the only limitation is what the system is capable of storing and transmitting. If you send your birthday to another person, the format does not matter as long as the person in the other end is able to understand it. "I was born on Christmas eve 1980" is completely understandable by a human.

However, people are smart and computers are dumb. For most applications, input has to be understood by a computer and the computer needs the information in some predefined format. The application has to tell the person entering data what it expects and can understand.

There are several ways to help humans and computers to understand each other better: Bound Input, Active/passive validation and Formatting.

Bound input

A Text Input Field can alternate between alphanumeric and numeric.

When input is constrained, for example to numbers only, the widget must indicate what the constraints are. People are ok with this arrangement, as long as it is easily to see and understand what the active constraints are.

Active Validation and Passive Validation

Active validation is done character by character as the user types.

Passive validation is performed after the entire input is complete. Either because the data has no meaning until the entire input is entered, or because validation has to be performed at the server.

A field that will be validated should visually indicate this.

Feedback should highlight the error and provide information about what the problem is.

A different approach. The user don't have to select the credit card type. As they type in their credit card number, the right card type is chosen (based on the card number). Looks cool, but I don't know if this has actual merit. Example from Masabi.

Formatting

Sometimes data is commonly displayed in one format, but stored inside the database in a different format.

An example could be a credit card number or a phone number. To make it easier for human beings to read a 16 digit credit card number, it is displayed in blocks of four digits. The human readable form of a date or phone number is probably not identical to the way it is stored in your internal database. Do not allow lazy developers to mess with this, make sure that the application displays data in the appropriate format.

It is worth stressing this issue because the problem is common. Companies print serial numbers or registration keys with spaces and hyphens but do not accept them when a user enters them into a form as they are printed.

The application can strip out any spaces or hyphens as the user types, or before validating.

Overwrite mode

Wherever appropriate, prefill form fields with your best guesses at the values the user wants. Overwrite mode selects all text, so it is easy to type in something else, without having to erase the suggested text.

For more about good defaults, visit Jenifer Tidwells pattern collection. Even better, buy the book.

Autocomplete

This is a large topic, it needs its own post, but the general idea is to guess what a person is about to enter and offer to complete the text. Combine a text input field, a popup list and some logic. Typically seen in URL address fields. Most decent mobile browsers offers autocomplete. This example is from Yahoo! Go:

Handling overflow

Text lines that are too long to display in full can be truncated. This is done by removing characters until the remaining text plus an ellipsis (three dots) fits the screen space. The ellipsis signals that data extends beyond the field boundary.

• Text should be right-truncated, i.e. characters are removed from the end of the string. 
• Numbers should be left-truncated, i.e. removed from the beginning.

Text that is too long can also be displayed as marquee text or auto-scrolling text.

The text moves toward left (opposite direction of reading) and is clipped at both ends. The text starts with a (1 second) pause and then moves smoothly to the left (about 7.5 characters per second). When the text is back to its starting point, it pauses again for 1 second before continuing.

Only the text field that has focus is scrolled, when the text looses focus, the text is immediately reverts to its truncated and static form.

Interface texts like labels or alerts should not be scrolled. And don’t scroll very long texts.

States

A general purpose text input field needs five editable states and five non-editable states.

Anything that can ease the pain

Text input is not straightforward on a mobile and anything that eases the pain of entering text is likely to make your application more successful.

There is many additional aspects not covered here. Especially if you work with touch-screen keyboards where you might be able to provide context specific shortcuts.

A good resource for more info is the Mobile Design Resources wiki at Little Springs design.

Muscle memory

This blogger has moved!

I want to invite you to my new gig over at Rift Labs. We are devloping Open Source Hardware for photographers. Come join us if you are into UX, design or photography at some level. Visit Rift Labs

 

This is the main entrance of the building I work in. Eh, to be precise it's the user interface to the door lock. (There are actually 2 user interfaces there. The door bell for the first-timers, and the panel below for "experienced users"). It's standard issue: Swipe your ID card, type in your code and the lock opens with a bzzzzz.

As you can see the numbers are worn off. They can't be read. How do people get in through the door? No they don't ring the door bell. But if they can't see the numbers, how can they enter their code? When I get to work each morning, there isn't a queue of 150 people outside failing to get the door to open. So they must be able to get the door to open somehow.

Muscle memory

We don't enter numbers, we leave it to our hand to type in the code. If we do it often enough, the hand learns the code, an effect called Muscle Memory. The hand does not remember the numbers, it remembers a movement pattern. Many people, when asked what their PIN code is, have to type it on a real or simulated keypad and look what numbers their hand type. In fact, when I asked my colleagues about the panel at the main entrance, none of them had even noticed that the numbers were missing!

The brain-to-hand connection is very well developed in humans. Manipulating something with our hand does not take much mental effort. As a result, direct manipulation feels effortless. The flip side of this is of course that if your software is inconsistent with learned behavior, your users will be frustrated.

Mobile menus

It's worth repeating that you should not design your software to be consistent across devices. You should design your software to be consistent with the device it runs on. Your users are unlikely to use your software on many different handsets. When your software runs on an old Sony Ericsson phone, it should use the hardware Back key down to the left. When it runs on a Nokia Series 40 it should use the softkey back to the right. Here are some common key layouts:

 

(Click to enlarge)

Common ground

Thankfully the handsets are converging towards this common behavior:

The panda and the blue button

This blogger has moved!

I want to invite you to my new gig over at Rift Labs. We are devloping Open Source Hardware for photographers. Come join us if you are into UX, design or photography at some level. Visit Rift Labs

 

A panda walks into a cafe. He orders a sandwich, eats it, then draws a gun and fires two shots in the air.

"Why?" ask the confused waiter, as the panda makes towards the exit. The panda produces a badly punctuated wildlife manual and tosses it over his shoulder.

"I'm a panda," he says, at the door. "Look it up!"

The waiter turns to the relevant entry and, sure enough, finds an explanation.
"Panda. Large black-and-white bear-like mammal, native to China. Eats, shoots and leaves.

Just like spoken or written language, visual language has its grammar. Without grammar, language would not make sense most of the time.  Without grammar, visual language is just as confusing.

Proximity is the Comma of visual grammar

In the above image we see three separate figures. In the image below we see the two figures on the right as a group, and the figure on the left as not belonging to the group. The only difference is that the figure in the center is moved a few pixels to the right. It's like putting a "visual comma" in between the first and second figure. The entire meaning changes.

(Images liberated from the infamous Artemy Lebedev)

Your brain is not passively receiving and storing visual input. The brain is actively trying to make sense out what you see, whether you want it or not. It can't be turned off. Designers who are not mindful of these principles can (and often do) confuse people.

Some of the rules the brain uses are described by the Gestalt Principles. If there ever was a visual grammar, the Gestalt Principles of Perception is it.

The sum is greater than the parts

The brain tries to give meaning to chaos, divine relationships between objects. The Gestalt approach emphasizes that we perceive objects as well-organized patterns rather than separate component parts. Here are some of the principles:

The Law of Proximity

We see objects close to each other as forming a group.

The Law of Similarity

We group elements if they are similar to each other.

"This is a great example of the gestalt law of similarity employed in interior design. [...] In this case, all of the dining room chairs are black. The shared color makes the chairs form a group based on similarity. The grouping effect is so strong that you really have to look closely to notice that the chairs are a mix of different designs."

(Gary Fisk, Applied Psychology)

The Law of Prägnanz (figure-ground)

Something is always "the object", the rest is "background". This effect is often used in optical illusions like the well known Rubin Vase above. Do you see the vase or the two faces?

In the Mac logo above, you either see the happy face frontal or a face in profile. In cases where two interpretations are possible, your brain can switch between them, but it can only see one interpretation at a time.

The Law of Symmetry

When we perceive objects we tend to perceive them as symmetrical shapes that form around their center.

A typical textbook example of the law of symmetry consists of a configuration of a number of brackets (the illustration above). When perceiving the configuration, we see three pairs of symmetrical brackets as opposed to 6 individual brackets, or two pairs and two singles. This happens despite what is suggested by some of the brackets immediate proximity to each other. (more here).

The Law of Closure

We perceptually complete objects that are incomplete.

The brain is constantly filling in the gaps in order to form whole objects out of separate component parts. Its surprisingly good at it; we identify patterns faster than the separate parts of the pattern. It might be an evolutionary issue. "Is that shadow over there a wolf preparing to attack me, or just a bush behind the three?" You want to make up your mind fairly quickly.

It is not only our visual system that does this. Our brain is a fill-in-the-gaps machine. Check this video interview with David Ariely.

 

Spot the errors:

Yahoo and Blackberry: The Yahoo! Flickr screen is quick to scan and easy to understand. To understand the Blackberry screen, you have to read every single word on the screen. What Gestalt Principles could the Blackberry designers use to make the screen better organized and less confusing?

AccuWeather: There is no way of knowing if the upper or the lower button is selected in the screen above (both states are shown). Is there a Gestalt Principle that the designers could have used to avoid the confusion?

Google Maps: What Gestalt Principle has the Google designers used to ensure that you don't confuse the search results with symbols on the map itself?

The quote is from the book Eats, Shoots & Leaves: The Zero Tolerance Approach to Punctuation by Lynne Truss. Apologies for punctuation errors in the text! :-D

Video demo of the Flat Music Player

This blogger has moved!

I want to invite you to my new gig over at Rift Labs. We are devloping Open Source Hardware for photographers. Come join us if you are into UX, design or photography at some level. Visit Rift Labs

 

I've done some more work on the Flat Music Player, and I'm happy to present to you a brief video demo of an over-designed grunge version, in splendid HD, for your video enjoyment! :-)

If you are seeing this in a reader or in an email, you may have to click through to the site to watch the video.

Recommended version: The best quality version can befound here. (8 mb)

Small version:

[UPDATE] There has been some sites reporting this as an upcoming product from Nokia. This is not so. I have no affiliation with Nokia. It is a concept design exploring spatial cognition, or in popular terms: A player that organizes music in piles, the way our brain prefers it, and not in lists, like a computer prefers it. You can find more about the thinking behind it here.

The iPhone SDK limitations has real causes

This blogger has moved!

I want to invite you to my new gig over at Rift Labs. We are devloping Open Source Hardware for photographers. Come join us if you are into UX, design or photography at some level. Visit Rift Labs

 

There has been a lot of huffing and puffing over limitations to iPhone 3rd party apps. Especially the "one app at a time" and "no background processes" rules. A lot of people claims that these particular restrictions does not apply to the native iPhone apps or to preferred partners that Apple lets under the tent. I would like to argue that this assumption is not correct.

Some even speculate that the reason is Steve Jobs' paranoia. I'm not privy to the mans mental state, but there is no need to search for psychologic explanations. There are real reasons for these limitations. Some hope for relief over the course of the beta period but it is unlikely that they will go away. At least on the current iPhone hardware.

 

Why no background processes?

The iPhone, as most smartphones, has a multi-chip architecture. OK, this is going to be oversimplified, but hang on. The iPhone has many CPU's and they are connected with and controlled by the central scrutinizer, the ARM 11 CPU that runs OS X. Each CPU has its own "OS", runs fairly independently and is optimized for very specific tasks. The baseband processor runs the signaling stack, the stuff that handles the communication to the cellular network. The audio codec handles the audio stream. The Wi-Fi transceiver deals with all the mechanics related to operating the Wi-Fi port. Etc, etc.

All high-priority real-time tasks run on dedicated CPUs. When needed, the main processor asks the other parts of the system to do their dance. When the music starts (!), the audio circuitry does all the hard work, and it does it much more efficiently than the general purpose ARM processor can.

When you are, say, calling, the main CPU doesn't really do much. After you have entered the number, it can pretty much idle until you hang up. (The chances of a 3rd party applications messing with the basic calling functions should be approximately zero unless they were allowed to initiate a new call or hang up in the midst of the current call.)

The system slows down, idle or shut down whatever CPUs that are not currently needed. The iPhones impressive battery life if achieved by carefully managing and slow-clocking the system. The internal hardware design philosophy in the iPhone is that the main CPU does as little work as possible and delegates tasks to specialized circuitry. If a 3rd party background process could run the main CPU constantly, this would drain battery in no time. Of course, a modern OS like OS X can deal with resources on a very detailed level, but the iPhone OS X is apparently not set up to police 3rd party apps.

 

Why only one app at a time?

Having several apps running at the same time would break the interaction paradigm of the iPhone. There is no way for people to handle multiple windows on the iPhone. iPhone apps does not have an Exit button. The way to exit an app is to press the home button which brings you to the home screen. This is the behavior that people expect and there simply is no mechanism to close an app and have the the previous app come to the foreground. The iPhone does not use the WIMP paradigm that desktop OS'es does. There are no Windows, no Menus, no Pointers. You can even argue that there are no Icons either (icons in the classic sense as "visual representation of objects you can manipulate"). This may be a bit strange for people new to mobile development.

 

Application silos

You can try the following experiments:

1. Send an SMS with a URL to the iPhone. In the SMS app, you can click the URL to open the browser, but the SMS application closes when you do. The only way to get back from the browser to the SMS app is to go to the home screen and open the SMS app again.

(Btw, Back is very rudimentary on the iPhone and there is no global history stack. See here for more about backstepping on mobile phones.)

2. If you open the Maps app from Contacts, the Contact list closes. In the Maps app, you can add to the contacts, but the Contact application is not really open, the Maps app has simply duplicated the Contact "view". The contact list view is recreated inside the Maps application. Maps can read and write data to the contact store, but Maps and Contacts are never open at the same time.

3. When you answer an incoming call, any app currently open is closed. When the call terminates, the app is brought back to the foreground but it has to reload any content. You can try this with Maps or the Safari browser.

If you pay close attention you can see that the native iPhone apps follows the same rules that are laid down for 3rd party apps.

I think its a bit unfortunate that the iPhone seem to have a strict "application silo" approach, but this architecture is probably the result of an overriding design goal of maximizing battery life. It is something 3rd party developers just have to live with. There is not really much multitasking going on on the iPhone. "One app at a time" is here to stay.

 

"The same API's as we use internally"

Any application running on a mobile phone should be prepared for immediate shutdown at any time. Best-case is you get a notification when it happens, but your app can be force-quit by the OS or by the user at any time. This is true for Symbian, Android, runtimes like Java ME, FlashLite, or WebKit. Windows Mobile may be an exception to this, I don't know WinMob too well.

If you want to create an IM application you need to store any incoming traffic on a server and any outgoing traffic on the phone. I guess we will see if my claims here are bogus when the AOL IM client ships.

It would be helpful if there was a way to request the iPhone to update the bullet for your application icon on the springboard (the home screen) with some regularity. There is obviously background processes running for receiving SMS and push mail notifications. (That does not mean that the Mail or SMS apps are running.) This notification mechanism ought to be available to 3rd parties also.

When Apple says that "it's the same API as we use internally to build iPhone applications" - this is almost correct :-) These limitations are based on hardware and software design choices and are not likely to go away during the beta period.

 

PS: Why not run everything on a single processor?

Apple could have chosen a different architecture where everything runs on the same processor. This is the architecture that Intel has been advocating for years (without much success). The advantage is lower chip count and lower BoM, but the main processor would have to run at a higher speed and the system would be less power efficient. Symbian has recently changed its architecture to allow single core designs and some low-end smartphones from Nokia now runs what is basically two operating systems on the same CPU.

DS

Backstepping on a mobile phone

This blogger has moved!

I want to invite you to my new gig over at Rift Labs. We are devloping Open Source Hardware for photographers. Come join us if you are into UX, design or photography at some level. Visit Rift Labs

 

Mobile applications have two ways to use the Back function; As "History" or as "A Path Home". This blog post discusses those approaches and backstepping in general.

Back as history:

Say I am in the phonebook and I click an address. The map opens. In the map I do a search and the search result includes an email address. I click it, the email editor opens and I send an email.

I have jumped from "application" to "application" in the phone. How do I want the Back-function to work? I'd want to backstep from the email to the map to the phonebook. This is the behavior we are used to from Internet browsers.

Back as "A path home":

On most phones the Back-key does not step back through a history. It takes you back to the main menu or idle screen. The advantage is that users never get lost and can fairly quickly get back to known territory.

The Home key

A prerequisite for the history stack approach is that the phone has a "Home" key. You don't want the user to have to press Back 20 times to get to the main menu. In addition to WinMob, Android and the iPhone uses a hardware Home key. Btw: In version 6 Windows Mobile changed the back behavior from "A path home" to a history stack.

If you want to have a history function in your application (assuming your application is large and/or deep), and the phone does not have a hardware Home key, what are your options? Your only choice is to give the right softkey double or triple duty as Back + Home or Clear + Back + Home key. It is doable, but has to be thought out carefully and it may not be worth the trouble.

How Back works in Browsers

Current Internet browsers use a stack to implement the Back and Forward button behavior: the effect is that a person can use these buttons to move up and down the single hierarchical path captured by the stack, but cannot use it to revisit any pages that branched off that path as these are popped off the stack.

To illustrate this problem, let us take a person who starts at page A and visits page B. He then clicks the Back button to return to page A, then visits page C. This navigation path can be shown as A → B ï A → C. The contents of the Back menu will now be {A,C}. Page B has been pruned out of the menu entirely. (source)

This behavior corresponds poorly to the mental model of most people. Most people are incorrectly thinking that the Back button models a history list of all pages visited. It may seem that this system is confusing and frustrating, but research reveals no major problems with stack based navigation! An advantage of Back navigation is that it represents a very light cognitive load. People tend to use a simple "click until the desired page is recognized" strategy, and are happy to trade "true back navigation" for this simple model.

Techniques more sophisticated than the stack could be employed, but the simplicity and the effect of conditioning from a decade of browser use, dictates that the Back navigation functionality should mimic the one seen in Internet web browsers.

Don't store everything

The following states in the UI should not be included in the history stack:

• Notifications 
• Confirmation dialogs 
• Login or password screens

The Back key

The right softkey is default back navigation. The ultimate Back (from the application perspective) is an exit to the idle state, (closing of the application). In this case the right softkey is labeled Exit instead of Back.

Many Sony Ericsson phones have a hardware Back key. Some phones have a Clear key that does double duty as Clear and Back key. The preferred behavior is that the control keys work the same way in the application as they do in the phone OS.

And note: Back navigation cannot be used as an Undo function. After backstepping to the previous page, the screen should look the same as before. The focus must be returned to the same items, and items  kept in the same positions on the screen. However, any data that is committed will not and should not be undone by "Back". An Undo function must be labeled "Undo" and must be UI widget-oriented and not page-oriented.

Back behavior in mBricks

We are contemplating this mBricks history stack behavior:

1. Navigation to a page pushes a page to the stack top. This is done by a call to historyPush(). historyPush() stores the page and the appearance of the page (focus, highlight and scroll). 
2. Using historyBack() or historyForward() moves the stack pointer down and up the stack respectively, displaying the page at that stack location. The actual stack contents are not altered when navigating with these functions. 
3. When navigating to a new page, any entries on the stack above the current position are popped off the stack, and the new page is added. Pages popped off the stack cannot be revisited using the historyBack() or historyForward() functions.

The mBricks history stack is page oriented. Populating the stack is controlled by the developer. All page visits are pushed to the history stack by default and the history stack has a 20 item limit.

The usability of Gmail for mobile

This blogger has moved!

I want to invite you to my new gig over at Rift Labs. We are devloping Open Source Hardware for photographers. Come join us if you are into UX, design or photography at some level. Visit Rift Labs

 

Hey Google, now that you have Google on the iPhone out of the way, could you take another stab at Gmail mobile? Here is a couple of issues that needs to be addressed.

Mobile User Experience according to Google

Leland Rechis presented Googles Mobile User Experience Strategy in a talk in NYC last year:

1. Understanding users, anywhere, anytime 
2. Fits in your pocket 
3. More personal than the PC 
4. Consistency across modes 
5. Localization is intensified 
6. Integrated devices, modes, products

Nothing revolutionary, but mostly sound principles. For a utility application like email the core is: Can the user get the information, and fast? Answer this question and you're far ahead of everyone else.

Gmail mobile is the kind of application that lends itself to a Activity-Centered Design. It certainly don't try to make the experience fun or interesting, the approach is: get the job done with a minimum of hassle. Gmail should be evaluated on these premises. Btw, I'm using version 1.5 on a N95.

The positive

Gmail mobile in general is a great app. Its a painless install, it starts reasonably fast, it is responsive and does not freeze your phone while it's retrieving mail. This is good. If you know Gmail from the fixed web, you recognize it immediately. Gmail organizes your mail in conversations. This is great, way better than organizing emails in folders. Gmail carries this over to mobile. The primary activities for most users are checking and reading email, and this main use case is well executed.

I managed this only once

The not so positive

The bad news is that Gmail stumbles when it comes to the secondary use cases: Searching mail and Composing mail.

Don't lose my text

Text entry on a mobile phone is hard for most users and one of the most important rules in mobile development is: "Thou shalt not mess up thy users data". Mobile use is fragmented. Suddenly the phone rings, the cab arrives, your bike tilts, you mistakenly hit the red button because a velociraptor bumped into you, whatever the reason.

It's easy to close Gmail. Hit the red button (End Call) and its gone. If you receive a phone call and hit the red button twice when you hang up, Gmail closes.

This is fine. What is not fine is that Gmail also loses the mail you were in the middle of typing. I think Gmail tries to save, but it's buggy. I managed to recover the mail once out of 10 tries.

Composing mails

Adding recipients is clumsy with several screens, a lot of clicking and separate functions for entering email addresses or selecting from "most contacted" or "all recipients". All of this could be done much more elegantly in a single screen with a continuous filter.

The built-in Address Book on my phone is full of email addresses, but there is no way add recipients from the Address Book. This not a technical problem, and it would be a useful feature in my case.

It's only possible to have a single draft. Mobile drafts and drafts in Gmail on the web is not related. I'm not sure I understand the reason for this.

Search don't work

Search Results Page

Conversations can grow long, especially if you subscribe to mailing lists, you can easily have 100 emails in a conversation.

If you search for something Gmail will return the conversation, not the individual email containing the word. It will tell you that "somewhere in these 100 mails we found your search term". Gmail does not highlight the hits in the text either, so you have to manually open each email and look for the text. Sort of ironic that Google can't get search right, but there you go.

A basic rule of an SRP (Search Results Page) is that it should display the results in an easily scannable way. The SRP uses a dual line list, and one improvement could be to display the search text in context on the second line. Another could be to use triple line lists. And the results should show the actual mails, not the first mail in a conversation.

Inconsistent use of UI controls

Adding recipients is clumsy

Who sets the standard anyway? If you make a desktop application, you are wise to follow the UI style of the OS, be it Mac, Windows or something else. The assumption is that users are familiar with how buttons, menus, windows etc. behave elsewhere on their computer so you are doing yourself and your users a disservice by introducing a different behavior.

The same goes for mobile. Many developers goes to great lengths to make their application behave consistently across different handsets. This is misguided. You should make your application consistent with the mobile OS it runs on. That means that your application will behave slightly different on different handsets.

Google tries to be consistent across handsets, and that means that Gmail won't feel completely home for anybody.

One example of this is that Gmail uses the left arrow key as a Clear key. Instead of using the Clear key that exist on the majority of phones, and implement a multimode right soft-key on phones that does not have it like Series 40. There is no phone in the universe that uses the left arrow key as a Clear key. Lazy mans solution, I say!

I guess a verdict would be that reading emails works great, the rest needs work.