Sender 11

Flat Music Player version 2

 

 

Click the image to see the animation (It opens in a new window).

Please note that the text below is necessary for understanding the animated example above. Well, I think so anyway :-)

The content itself is the interface

I've been thinking a bit about "content is the interface" lately. The idea is to avoid deep hierarchies of thin information by intensifying the information resolution on the screen. Edward Tufte is the great proponent of this idea and lately it has been made current by the MEX conference as their Manifesto #1: Content itself will be the interface of the future. 

It is easy to see how to employ this paradigm when the information is manageable, when it's on a "human scale" so to speak. The in-flight entertainment selection in an airline seat, the variety of coffees in a coffee shop, the selection of cars at a dealership. You can take in the information and hold it all in your head at once. (Deciding what to choose is of course a different matter…) 

It is certainly both possible and advantageous to use the content itself for most navigation and interaction tasks. But when I tried to apply "content is the interface" to music there were a few additional issues. A typical device can store 8GB of music; that is 2000 songs or about 250 full albums. Many devices can store a lot more. Not to mention the millions of songs you can browse when you want to buy music. Presenting all this music directly is not possible. 

In addition, music is sound; you can't actually see it or touch it. You have to choose a visual representation and I've used album art because that’s an abstraction that most people understand and are used to. With digital distribution, there is some movement away from albums in favor of individual songs. I've considered using images of the artists as visual representation but it does not work well. Album art has a strong visual identity and is associated with a particular selection of songs. Is scales well; you are able to make out the album even when scaled way down. This is not the case for photos of artists or bands. They change randomly over time and they don't scale well. 

Spatial reasoning

In addition to this, I wanted to try to take advantage of spatial reasoning and spatial memory to make it easier to find and navigate stuff. Let the user see the scope of information available. Start by showing  the big picture. When it makes sense, let it behave more like real-world objects. You can normally pick up objects where you left them off. They don’t move when you are not watching, something digital objects often do. (Insert your favorite joke about spouse here.) Over the last couple hundred thousand years our brains has developed a fantastic ability to take in and store where stuff is in our immediate surroundings. Since mobile screens are a part of our immediate surroundings, we should try to take advantage of this ability. It might sometimes make user interfaces a bit less confusing. 

This is version 2 of the concept player that I fiddled with over at OTA. This time I've used a larger touch screen (the size is equal to the upcoming Nokia Tube). 

Current music players

 

Screens from a Nokia Series 40 phone and the Sony Ericsson 960W phone. 

The problem with the illustrated players is that the vast majority of the screen is décor and "computer administrative debris". The information value is extremely small. All that fits on the 240 x 320 pixel screens are three band names (Nokia) or four song names (Sony Ericsson). 

Nearly all players use a hierarchical presentation where the user steps through levels; Genre, Artist, Album, Song. The icons are genetic and do not add any information value. The layout of the W960 music player only uses half the screen width for the song name. The result is that most song names have to be truncated even if they are rendered in a small font. 

Playlists

Terminology problem alert: Playlists are normally used to describe a (small) selection of songs on your device. For lack of a better word, I also use playlist to mean the collection of all songs on your device including their metadata. 

Going forward I expect that playlists may become more important than the music files themselves. Renting (subscription) may well be the way to consume music in the future, and the list of your music will be more important than the files themselves. A development similar to how bookmarks has moved away from individual browsers and computers and up in the "cloud" from where they are accessible regardless of device. Obvious example: del.icio.us. If you change your music subscription from, say, Rhapsody to Napster, you just take along your playlist and continue enjoying your music. 

I don’t think there should be any significant difference between a streaming player and a file player. If you add an album, it should appear in your music collection the same way as empeethree files does. ("Internet radio" streaming is different of course, as you don’t select music, you select a station.) For performance reasons subscribed music probably have to be stored or cached locally. 

Hardware

These design sketches are made with the nascent/emerging trend of high resolution widescreen touch phones in mind. Examples are Sony Ericsson Xperia, Nokia Tube and Apple iPhone. 

 

Note that the sketches focuses on the interaction model, not on the feature set or graphic design. Also, the intention has been to work on something that might be useful on actual emerging hardware, so no Jetson style 3D holographic interaction models at this time. Most of the sketches are so called Hi-Fi sketches because it's easier to convey the ideas with something that resembles a product than with plain wireframes. 

The basic structure of the app

 

• The layer named “Albums” holds the music library. It is divided into rectangular sections (labeled Indie, R&B etc…) 
• "Hype" is a closable panel. It has an, um, "river" where info related to the music floats by slowly. It has a drop area where you can drag and drop content that you want to subscribe to. (No, let's not call it the shore or the riverbank :-)) 
• "Shop" is another closable panel that contains a music shop.

Albums (the library)

 

The busy impression of this well filled music library is intentional. The library contains about 200 albums. (And movies. I'll stick to music to avoid distractions, but I don't see any reason a movie player needs to be distinct from a music player.) 

• No particular distinction is made between music that are streamed and music that are files. 
• The albums are not sorted alphabetically. New and recently played albums appear on the top of each pile. The reasoning behind this is that the human brain understands space well. We know approximately where things are when they have a position in a physical space. If you want play a song you tend to remember that it was "somewhere in the middle of the pile over there". 
• Albums that are played often grow bigger in size. 
• Classification (Rock, Indie, R&B etc.) is based on metadata for the music. The user can change classification to fit personal preference. Missing album art is automatically retrieved from relevant web services. It is also possible to identify unknown music tracks via something like Gracenote. 
• Suggestions are based on playing patterns and/or suggestion engines like last.fm, Pandora and social engines "people you know listens to…".

The bottom part of the screen displays a bar when music is playing. It holds player and volume controls. If the hardware has separate player and volume controls, that's good. That will make it possible to control the player when in the pocket. 

The top part of the screen needs a back button and probably a search function (not shown). 

 

Clicking on a category zooms/expands the category to fill the screen. I have show the albums organized in a grid in this example. But keeping the position and size-difference when zoomed-out needs to be investigated.

Clicking on an album opens it and starts playing.

Individual songs 

 

The player needs a mechanism to access individual songs. Tapping the album above the play-control in the lower left corner could pop up a more detailed player.

Discovery

The closable panel labeled "Hype" is for exploring. Discovery of new artists and bands is an important function. 

• Contains related hype (blog posts, news, etc) about the artist or band. 
• Can be accessed while listening. 
• Displays information related to the currently selected music or to music in general if no particular album is selected.

• The hype blurbs or "story boxes" flows down from the top. 
• You can subscribe to any of the sources you find there by dragging them over to the leftmost column. 
• I've used last.fm to illustrate one source of info, but sources can be anything, HTML-based, API-based, RSS based, etc.

Accessing hype content 

 

Tapping any story box in the Hype panel zooms to the full content. 

Shop

 

The Shop should display its content in a nice and tidy fashion, as shops do. Since it's so tidy, gridlines between sections don't seem necessary. Navigation should probably be a combination of Tap-to-zoom  and Drag-to-scroll. I haven't spent a lot of time on the shop to be honest. 

End notes

The Flat Music Player version 2 ended up having only play/pause controls, a back-button, and two closable panels that are manipulated by gestures. An actual product would need a little more; it has to be possible to type in names for sections, probably URLs, and ways to connect to friends. But I think this shows that you can get quite far using the content itself as the interface. 

I was looking for more plasticity and viscosity than I could wring out of Flash, hopefully the animation gives some sense of the intent.

[UPDATE] I entered the The Flat Music Player in the MEX Design Competition. If you like it, you can vote for it.

Inspiration

Lumines
Amaztype
last.fm
Hype Machine
Universe by Jonathan Harris
NokiaDesign

More on mobile screen size trends

I recently wrote about screen size trends and the trend is clear: 240 x 320 (aka QVGA) is the new baseline screen size, both for feature phones and smartphones. Smaller screen sizes like 176 x 220 is disappearing and the 128 x 160 size is pushed down into basic phones.

QVGA will be the new small size, but it is not clear what the new dominant large screen size will be, or if there is going to be any.

What is pretty clear is the trend towards widescreen and landscape orientation for high-end phones. QVGA and larger phones are getting automatic screen rotation. Landscape is often preferred for media consumption and handheld movie consumption has been "promising" for quite some time.

The primary design objective for the iPhone (in my opinion) was to make the optimal video iPod. When you have 70% marked share you need to grow into new markets. Several new phones from other manufacturers seem to have the similar design objectives.

I have compared some available and coming devices that could represent a trend. LG and Samsung seems to be partial to the 400 x 240 screen size. They have released several devices with this screen size the last year or so. Nokia has recently tipped their hand with the Nokia Tube. The name obviously refers to a video use-case. At MWC in Barcelona in February they showed a demo that ran on a 360 x 640 screen. This gives a 16:9 resolution, the same as HDTV. 640 x 360 is also called QHD for "Quarter HD".

The Sony Ericsson Xperia has been developed for the US market specifically. It runs WinMob 6.1 on a 800 x 600 screen, a resolution not supported in any official Microsoft documentation, but I guess Microsoft saw fit to go the extra mile in this case :-) The Xperia has an extreme resolution, it has more pixels on 3 inches that a good ole VGA desktop screen. I'm not sure that the Xperia exact screen size and resolution represents a trend, is kind of an extreme case. But it underscores the general high-end widescreen trend.

Pixel dimensions and aspect ratios

I used 9 as lowest common denominator for ease of comparison.

Physical screen size

Most of the devices uses 3–3.5 inch screens, something that does not leave much space for a keyboard. The phones are either clamshell or pure touch. They are also fairly similar in physical size.

Gestures, pixels and bandwidth

After years of designing for really small screens and always in portrait orientation, these devices sure feels like drowning in pixels. I absolutely expect to see some great and innovative user interfaces coming out of gestures, pixels and bandwidth!

[Note:] No mention of Motorola here, for the simple reason that I have no idea what they are up to.

[Edit] Just to be clear, the reason I'm exited is because the phones above makes it possible to design for a wide screen. I'm aware that many or most of the standard functions in most of the phones above is intended for vertical use.

Mobile screen size trends

Over at mBricks my colleagues have put a lot of work into the device database (we work with WURFL data and contribute back as well as we can). I took the opportunity to take a closer look at screen size trends. The data I have covers about 400 different device models sold from 2005 to today.

This shows the the most significant screen sizes, from the smallest to the largest. I have added a couple of upcoming phones as well, they are the ones with the dotted lines.

Over the years the relative screen size difference has increased. The difference between the smallest (128 x 128) and the largest (800 x 480) is now a factor of 23. That means the largest screen is 23 times bigger than the smallest one.

You can see that the smaller screens have a portrait orientation and the large screens have a landscape orientation. Between them are the phones that can change orientation, they can work in both landscape and portrait. 240 x 320 is the dominant screen size overall.

Resolution

I did a rough dpi (ppi to be exact) calculation for some popular phone models. The pixel density actually increases when the pixel count increases. The screens are not only getting bigger, they are getting sharper at the same time.

There is an upper limit to what dpi is meaningful. At a certain density, the eye can no longer see any difference. If the specs are correct, the upcoming Sony Ericsson Xperia X1 will have a pixel density of 298. That is the highest I've seen on a mobile phone yet. The human eye can resolve about 340 dpi at one foot viewing distance IIRC, but tests show that most people don't see much difference between a 150 and a 300 dpi image. So 298 dpi should be plenty.

Unfortunately, for LCD screens, increased pixel density doesn't give us more brightness. More brightness makes the screen easier to read outdoors and is more important than resolution from a usability perspective. OLED displays will help with this, but we are drifting off topic.

 

Yes, a grand total of 26 different screen sizes. I only counted phones that had a color screen, ran Java and had a browser.

As you can see, just a handful of variants makes up the majority of phones. Lets take a look at them:

 

It is obvious that 240 x 320 (also called QVGA) is on a roll. It is by far the most common and it is growing rapidly. If you develop, this should be your target screen size.

 

When you develop, you primarily need to worry about the width of the screen. For most practical purposes, the height takes care of itself. I have lumped together 176 x 208 and 176 x 220 for this graph. There is still a lot of them in the market, but their numbers have been decreasing since January 2007. In 18 months this screen size may be out of the market. Nokia haven't launched a phone with this screen size in 1,5 years. I have also lumped together 128 x 128 and 128 x 160.

Phone screen sizes has had a tendency to come in pairs. Each manufacturer had their own variation on large-screen high-end models and small-screen low-end" models.

Manufacturer	small screen	big screen
Sony Ericsson	128 x 160	176 x 220
Nokia	128 x 128	176 x 208
Samsung	128 x 160	176 x 220
Siemens	130 x 130	132 x 176

 

Individual variations is disappearing, the trend is:

Manufacturer	small screen	big screen
Everybody+dog:	240 x 320	?

 

The majority of odd-sized screens are disappearing. No phones with the above screen sizes have been introduced for at least 4 quarters.

The dead and dying are partly BenQ-Siemens (who has left the market), partly the old Sony Ericsson P-series and partly the Nokia hires versions: E60/70 and N80/90.

The odd ones

What about the other ones, the not so popular, but not dying? These are the ones that do you in.

Fashion phones:
128x220	Samsung F210
240x400	LG favorite. Prada and Viewty.
Handheld touchscreens of the iPhone variety:
240x440	Various Palm and HP
240x480	LG KF700
Typical Palm/Blackberry form factor. US enterprise with portrait or square displays:
240x240	Samsung F210
240x260	Blackberry Pearl 8100
320x240	Various
320x320	Palm and Samsung
Clamshell:
640x480	HTC X7500, Qtek 9000, etc.
800x352	Nokia E90 Communicator
800x400	Sony Ericsson Xperia X1

 

Future

What will be the dominant screen sizes in the future? Well, the 128 width screen sizes are moving down from feature phones to basic phones and there are very few manufacturers that still uses 176 width screens. It looks like 240 x 320 is the new base line.

What will the dominant large screen size be? Let's hope there will be one. Right now there are a lot of different ones, every manufacturer has their own and I can't see a clear trend. The two hottest form factors right now is the handheld touch screens (ala iPhone) and the QWERTY clamshell (ala HTC).

For the handhelds, the 320 x 480 iPhone size is a possibility. It has an ok resolution for a 3,5 inch size. The Nokia "Tube" phone will have a 360 x 640 size (my estimate) and is another possibility.

For the clamshell form factor, 640 x 480 screen size would be my bet.

About the data

The data includes all the 400 different phone models sold in the Norwegian market from 2005 to 2008. I've given them four quarters of life time past the quarter they were introduced. That means an average of 14,5 months. This might be a little short.

I have left out basic phones. Only phones that has a color screen, support for java and a browser are included.

The Norwegian market is not very different from most other European markets. It's GSM only, the majority of handsets are sold with 12 month operator binding. Sony Ericsson, Nokia and Samsung are market leaders. Motorola is weak, HTC is up and coming in the enterprise market. The data includes the iPhone even though it has no official distribution. There is quite a bit of "black" iPhone imports.

Reliable sales figures for each individual device is not available, so the data is not weighted on popularity. But I don't think popularity would give a very different conclusion. I've checked "top ten" lists where available and none of the "odd" screen sizes seem to be big sellers.

Note: all sizes are written as width x height. So a 240 x 320 screen is portrait while a 320 x 240 screen is landscape.

Flat Music Player

I'm at the two day Over The Air conference in London. It's a combined code camp and conference. As is customary, there is a hackathon going on. Yesterday I was sitting in a couple of sessions bored (not all of the sessions are good to be frank) and I thought I'd join the competition and submit a design. I don't know if something that doesn't run code is admissible at a code camp... we'll see.

Here is the submission:

The problem

The majority of current music applications on mobile phones are surprisingly poor.

    

Screens from a Nokia Series 40 phone and the Sony Ericsson 960W phone.

The problem with the illustrated players is that the vast majority of the screen is décor and "computer administrative debris". The information value is extremely small. All that fits on the 240 x 320 pixel screens are three band names (Nokia) or four song names (SE).

Nearly all players use a hierarchical presentation where the user steps through levels; Genre, Artist, Album, Song.

The icons are genetic and do not add any information value. The layout of the W960 music player only uses half the screen width for the song name. The result is that most song names have to be truncated even if they are rendered in a small font.

Flat Music Player

This is an attempt to apply a different approach than the hierarchical lists used in music players in most current phones. 

The assumption is that people do not think about music in hierarchical manner. People organize their music according to their own head, primarily in a spacial image.

I don’t think there should be any significant difference between a streaming player and a file player. If you add an album, it should appear in your music collection the same way as empeethree files does. ("internet radio" streaming is different of course, as you don’t select music, you select a station.)

I wanted to design this so it could theoretically be implemented for current phones on the market today. That means 240 x 320 screen size, both touch and softkeys and a layout that can be adopted for both portrait and landscape.

I’ve tried the following:

• Use the content itself as the interface
• Flatten the information space somewhat
• Provide for spatial reasoning and spatial memory

Main view

The job of the main view is to give you a complete overview. It is based on album art because album art is (currently) the best visual representation of the music. At this zoomed-out scale, the albums themselves are tiny color blurbs, but you can still recognize the individual albums.

The albums are not sorted alphabetically. New albums appear on the top of each section. The reasoning behind this is that the human brain is good with space. We know approximately where things are in space. If you want play a song you tend to remember that it was "somewhere in the middle of the Indie section".

A search is started merely by typing from the keypad. Search should be modeless, i.e. searching artists, album titles and songs at the same time. It might even get fancy and return music from the sixties if you search for "1960".

Classification is done automatically. The user can change or override classification.

Suggestions are generated automatically.

 

The sketch is charcoal, or shades of gray (standard designer issue), but it must be possible for the user to add and change around colors. MySpace-ification is admissible :-). The size and layout of the sections should be user-adjustable.

Zooming interface

Each section is selectable. When clicked or tapped on, the section expands or "zooms" to fill the entire screen. 

In this view, the albums can be scrolled and individual albums can be selected.

I made the text too small unfortunately, it has to be bigger, but must fit the same grid.

Playing

Clicking on an album opens the album, displays the song list and first song is ready to play.

The progress bar is shown under the song name, so you see what the next or previous songs are. 

This view should also offer means to discover other music "like this", bands that have influenced this band, followers, etc, etc.

The play/pause player controls are mapped to the joystick push. Fast forward and rewind is mapped to the joystick right and left. The concept of "Stop" has no use on a digital player. 

Sharing

I didn’t have time to do a "hifi" mockup of this screen.

The idea is that the recipients on top are "near", for example Bluetooth devices nearby. The recipients below are "further away", they are IM friends, Facebook, Address Book contacts etc.

Browsing new music

      

Straightforward horizontal scrolling interface. You can switch between a song list and and band/album information. 

It turned out looking like it’s more or less lifted from iTunes, I’m sorry.

Needs work, but time is up. I have to submit now.

[Update]

Lots of cool demos here. I especially liked the Social Network Open Butler (presented by Robert "Jamie" Munro). It aggregates data from several social networks into your mobile phone address book. It can add a picture found on Facebook and a place from, well, from somewhere else :-) I'll link to it if I can find something posted later.

Phone Phight by Russ Anderson is a game in the starwars genre where two people battle it out using their Nokia N95s as "lightsabres". The phones are bluetooth connected and uses the game uses the accelerometer and bluetooth to decide who wins. Hopefully someone caught it on video!

[Update]
Here is a video showing Phone Phight:

Administrative debris

It occurred to me today that this blog is one year old. Hey, it's an anniversary! Actually it was a couple of days ago, but anyway.
I guess I'd like to use this opportunity to add a bit of administrative debris to this blog.

The thing that prompted me to start the blog a year ago was a lack of resources on Interaction Design applied to Mobile. My plan, if I had one, was to write down my thoughts about Interaction Design in general and Mobile Interaction Design in particular. I mean, If I can't read what I write, how can I know what I think? That is not my saying, it sounds like a Mark Twain quote, I'm not sure. But I think it's very appropriate for me. I spend some of my time deconstructing user interfaces, trying to reverse engineer what thinking (if any) that went into a particular design choice. From time to time I see really brilliant work, but most of the time investigating UIs mainly provides clues to why the design fails. And understanding failures is as educating as understanding successes. Writing helps me organize myself. If it can benefit anyone else, it makes me very happy.

Moving to London

Come June we are moving to London. "We" means my wife, me and our toddler. I'm really excited about this! There is a lot of mobile innovation and development going on in London and I look forward to being a part of that.

The aviator is not me

From time to time I meet people that I can't recall having met before and they may say things like "I didn't recognize you with the beard". I think it might be the aviator in the blog banner. The aviator is metaphorical. I should probably update the About page. If you want to know what I look like check the buddy icon on for example flickr.com/mhjerde.

The future

I plan to write more about hardcore mobile IxD issues. Apart from that I don't have any great changes planned: There is not going to be any product reviews, if I write about a particular product, its usually from a IxD angle. There is no regurgitated news here. I may comment on industry news/events tho. I probably post 2-3 times a month. If you like the blog, the good news is that it's not going to change much. If you don't like it, the bad new is that it's not going to improve much either.

On that note, my advise is: Don't read too many blogs. Spending time sketching, thinking and talking to your users is the best use of your time. If you need to organize your thoughts, write. I can't offer you any advice on writing, but if I recall correctly Mark Twain has one: Write drunk, edit sober. He-he, might work. Just don't skip the last part. Get your ideas down on paper, into Photoshop, Illustrator, Eclipse, Visio, whatever your tools are. Nothing gets realized as long as it is only inside your head. If reading blogs and news takes away from that, unsubscribe today, including this blog.

See you.
Morten

The iPhone SDK limitations has real causes

There has been a lot of huffing and puffing over limitations to iPhone 3rd party apps. Especially the "one app at a time" and "no background processes" rules. A lot of people claims that these particular restrictions does not apply to the native iPhone apps or to preferred partners that Apple lets under the tent. I would like to argue that this assumption is not correct.

Some even speculate that the reason is Steve Jobs' paranoia. I'm not privy to the mans mental state, but there is no need to search for psychologic explanations. There are real reasons for these limitations. Some hope for relief over the course of the beta period but it is unlikely that they will go away. At least on the current iPhone hardware.

 

Why no background processes?

The iPhone, as most smartphones, has a multi-chip architecture. OK, this is going to be oversimplified, but hang on. The iPhone has many CPU's and they are connected with and controlled by the central scrutinizer, the ARM 11 CPU that runs OS X. Each CPU has its own "OS", runs fairly independently and is optimized for very specific tasks. The baseband processor runs the signaling stack, the stuff that handles the communication to the cellular network. The audio codec handles the audio stream. The Wi-Fi transceiver deals with all the mechanics related to operating the Wi-Fi port. Etc, etc.

All high-priority real-time tasks run on dedicated CPUs. When needed, the main processor asks the other parts of the system to do their dance. When the music starts (!), the audio circuitry does all the hard work, and it does it much more efficiently than the general purpose ARM processor can.

When you are, say, calling, the main CPU doesn't really do much. After you have entered the number, it can pretty much idle until you hang up. (The chances of a 3rd party applications messing with the basic calling functions should be approximately zero unless they were allowed to initiate a new call or hang up in the midst of the current call.)

The system slows down, idle or shut down whatever CPUs that are not currently needed. The iPhones impressive battery life if achieved by carefully managing and slow-clocking the system. The internal hardware design philosophy in the iPhone is that the main CPU does as little work as possible and delegates tasks to specialized circuitry. If a 3rd party background process could run the main CPU constantly, this would drain battery in no time. Of course, a modern OS like OS X can deal with resources on a very detailed level, but the iPhone OS X is apparently not set up to police 3rd party apps.

 

Why only one app at a time?

Having several apps running at the same time would break the interaction paradigm of the iPhone. There is no way for people to handle multiple windows on the iPhone. iPhone apps does not have an Exit button. The way to exit an app is to press the home button which brings you to the home screen. This is the behavior that people expect and there simply is no mechanism to close an app and have the the previous app come to the foreground. The iPhone does not use the WIMP paradigm that desktop OS'es does. There are no Windows, no Menus, no Pointers. You can even argue that there are no Icons either (icons in the classic sense as "visual representation of objects you can manipulate"). This may be a bit strange for people new to mobile development.

 

Application silos

You can try the following experiments:

1. Send an SMS with a URL to the iPhone. In the SMS app, you can click the URL to open the browser, but the SMS application closes when you do. The only way to get back from the browser to the SMS app is to go to the home screen and open the SMS app again.

(Btw, Back is very rudimentary on the iPhone and there is no global history stack. See here for more about backstepping on mobile phones.)

2. If you open the Maps app from Contacts, the Contact list closes. In the Maps app, you can add to the contacts, but the Contact application is not really open, the Maps app has simply duplicated the Contact "view". The contact list view is recreated inside the Maps application. Maps can read and write data to the contact store, but Maps and Contacts are never open at the same time.

3. When you answer an incoming call, any app currently open is closed. When the call terminates, the app is brought back to the foreground but it has to reload any content. You can try this with Maps or the Safari browser.

If you pay close attention you can see that the native iPhone apps follows the same rules that are laid down for 3rd party apps.

I think its a bit unfortunate that the iPhone seem to have a strict "application silo" approach, but this architecture is probably the result of an overriding design goal of maximizing battery life. It is something 3rd party developers just have to live with. There is not really much multitasking going on on the iPhone. "One app at a time" is here to stay.

 

"The same API's as we use internally"

Any application running on a mobile phone should be prepared for immediate shutdown at any time. Best-case is you get a notification when it happens, but your app can be force-quit by the OS or by the user at any time. This is true for Symbian, Android, runtimes like Java ME, FlashLite, or WebKit. Windows Mobile may be an exception to this, I don't know WinMob too well.

If you want to create an IM application you need to store any incoming traffic on a server and any outgoing traffic on the phone. I guess we will see if my claims here are bogus when the AOL IM client ships.

It would be helpful if there was a way to request the iPhone to update the bullet for your application icon on the springboard (the home screen) with some regularity. There is obviously background processes running for receiving SMS and push mail notifications. (That does not mean that the Mail or SMS apps are running.) This notification mechanism ought to be available to 3rd parties also.

When Apple says that "it's the same API as we use internally to build iPhone applications" - this is almost correct :-) These limitations are based on hardware and software design choices and are not likely to go away during the beta period.

 

PS: Why not run everything on a single processor?

Apple could have chosen a different architecture where everything runs on the same processor. This is the architecture that Intel has been advocating for years (without much success). The advantage is lower chip count and lower BoM, but the main processor would have to run at a higher speed and the system would be less power efficient. Symbian has recently changed its architecture to allow single core designs and some low-end smartphones from Nokia now runs what is basically two operating systems on the same CPU.

DS

2D navigation on a mobile screen

Navigating large information spaces can be disorienting even on a large screen. On a small screen it should ideally be avoided, but that is not where we are headed. Increased storage capacity and higher bandwidth mean that you can have thousands of songs, images, messages and whatnot on your phone. We need good ways to interact with all this data. This post looks at 2D navigation and investigates what techniques good mobile browsers employ to make it easier to interact with large web pages.

2 dimensional content

When designing for mobile phones you often encounter situations where the content you need to display exceeds what can be shown on a single screen. For example an email message. The obvious solution is to format the text to the width of the screen and allow people to scroll the text up and down (vertically). You may choose sideways (horizontal) scrolling to let people interact with image galleries or wizards. Vertical scrolling works, horizontal scrolling works. But horizontal and vertical scrolling at the same time does not work. People get lost almost immediately.

There is one case in particular where this 2-way scrolling problem raises its ugly head, and it's called the Web. People want to be able to browse the Internet on their phone, but most web pages are meant to be displayed on much larger screens. Human eyes just can't read a page designed for a 19" screen when it's displayed on a 3" screen.

From an interaction perspective solving the "web browsing problem" is particularly hard. As pointed out elsewhere, a web browser is not exactly a natural for a mobile phone. "It would be hard to create a computing architecture more inappropriate for use over a cellular data network." (Michael Mace)

Good examples

I think it would be interesting to look at two approaches to the 2D scrolling problem. The Safari on iPhone because it currently offers the best web experience on a mobile phone and Opera Mini because it has some interesting tricks up its sleeve and it works really hard to make the web usable on very small screens.

The two browsers represent different philosophies:

Apple: Adapt the phone to the web.
Opera: Adapt the web to the phone.

Apple makes both the hardware and the software and has only one screen size to worry about. The iPhone uses direct manipulation, the user controls the browser directly with finger gestures like tap, double tap, drag and pinch. This gives a great sense of control.

Opera arguably has a harder job. Opera Mini runs on many different screen sizes, and it works on phones that has indirect manipulation. Since it's a Java ME application there are additional technical restrictions, for example the size and type of fonts Opera Mini can display.

Direct vs. indirect manipulation

On a direct manipulation device like the iPhone, you just tap anything directly.

On indirect manipulation devices, one item on the screen always has focus. You use the joystick as an intermediate device to move the focus to the item you want and click it. If the screen has 20 focusable items, for example 20 links, you may have to step through 19 links to get to the one you want.

Because of this, prioritizing is very important on a indirect manipulation device. You want to put the most important or most used stuff on top so the user has to do as little clicking and scrolling as possible.

But as we know, web pages are not designed in such a prioritized manner. The top area is usually filled with a number of links for general navigation, ads, search, etc, etc.

Example: The New York Times page shown to the right. The top headline is the 73rd link on the page. There is about 350 links before you come to the "subscribe to RSS" link. There is about 375 focusable items in total including images, forms, buttons etc (yes, I counted them).
No one is willing to step through hundreds of links. You need a strategy that makes it possible to reach any link on the page without stepping through them all.

Lesson 1: Zoom instead of scrolling

The main trick to solving the 2-way scrolling problem is to avoid it entirely. Both Safari on iPhone and Opera Mini encourages you to zoom in and out instead of scrolling. Scrolling both vertically and horizontally is disorienting. But if you shrink the content to fit the width of the screen the user won't get lost.

On the desktop, a browser usually has size and resolution enough to both display the entire width of the page and show the text in a readable size. In a phone browser it is necessary to zoom in to read and zoom out to navigate. There is basically two views for each page. The "Overview" where you navigate around the screen and the "Detail View" where you read or click links. The purpose of the Overview is avoid getting lost. Its like in a film. A scene often starts with an establishing shot, an overview of the scene before going to a close-up. The establishing shot tells the viewer that "we are now back at the mothership" and "here are the people in the scene". Without the establishing shot, the viewer might think that we were still down on Caprica.

Dual views of course require additional user interaction. In addition to scrolling, reading and clicking links, the user has to zoom in and out as well. You would be mistaken to think this bad a thing. This zoom in/out mechanism is greatly preferred by users compared to a keyhole experience. Just check how people are praising Safari and Mini. The number of clicks is not the most important parameter for user satisfaction.

Lesson 2: Throw pixels at the problem

The iPhone screen has twice as many pixels as a typical smartphone. (The screen is not physically twice as big. It is bigger, but the pixel density is also higher than most other phones.)

Pretty much all other mobile browsers are "keyhole browsers". They show a tiny part of a bigger page. Even Nokia's version of the WebKit browser takes this approach by default. You don't want the keyhole effect. Both Safari on iPhone and Opera Mini avoids the keyhole effect by showing you the entire page as default. The more pixels you have the better you can do this. Thanks to the high resolution of the iPhone, you can often read headlines and this makes it easy zoom in on what you want to read.

Safari on iPhone also attempts to provides a "pixel perfect" rendering of the page. (I know, there is no such thing as pixel perfect when it comes to browsers...).

Opera Mini runs on any screen size and can't depend on pixelcount, so it displays a rough representation. Even though you can't read the text, you see headlines, images, text columns etc. Enough to know approximately where to zoom in. This works because after two decades of conditioning, people know what the usual elements of a web page is. They have a "web page schemata".

Opera Mini renders most all text and images in columns that will fit the width on the screen when you zoom in on it. Actually, it narrows text and images that is too wide and leaves the rest as it is.

This sometimes results in "gaps" in the page overview. The iPhone displays the layout perfectly, but sometimes the text is too wide and does not fit the screen when you zoom in.

We have another philosophy difference:

Apple: If we need to sacrifice anything its certainly not going to be the web designers work!
Opera: Never mind the pixel-police, readable text is more important!

 

Lesson 3: Render for a small screen

How wide is the World Wide Web?

• The Web is currently 1024 pixels wide 
• The iPhone is 320 or 480 pixels wide 
• Smartphones are 240 pixels wide 
• Small Feature Phones are 128 pixels wide

Small Screen Rendering basically means creating a version of the web page in one single column and as wide as your mobile phone screen. Here is an example of how Opera Mini reorganizes the web page to make it easier to consume it on a small screen. 

The page has a fairly simple structure, chosen for clarity. As you can see, Mini decides what the header is, what the footer is, and what the main column is. It stacks the content to the best of its ability and scales images to fit the phone screen. (I you want a closer look at Small Screen Rendering, press Shift-F11 in the desktop version of Opera to view any web page with Small Screen Rendering.)

Safari on iPhone does not reorganize the screen, it simply zooms in on the area of the screen that you double tap on. If the column is wide and the text is to small to read, you can tilt the phone to horizontal.

Lesson 4: Follow the flow

Both Opera Mini and Safari on iPhone "snaps" to the column. Opera Mini even has this cool effect where it follows the column even when it's not straight. The browser slaloms or snakes down the column.

The magnetic or "gravitational" effect is especially important on the iPhone where you scroll by dragging or flicking. Without it, it would be very easy to scroll outside the column and lose a bit of the text on one or the other side. None of the browsers prevents you from scrolling away from the column, they just help you stay on track.

Other Zoomable User Interfaces

There is a number of other approaches to the problem of navigating large information spaces. There is an amazing demo of Photosynth at TED. Mobile examples are Deepfish, Google Maps, Zumobi, etc.

Roadmap, what Roadmap?

 

January 2007:

"I don’t want people to think of this as a computer. We define everything that is on the phone."

Steve Jobs to New York Times

 

June 2007:

"We’ve come up with a very sweet solution. You’ve got everything you need, if you know how to write apps using existing Web standards"

Steve Jobs at Apple WWDC 07

 

Match 2008:

"Starting today, we're opening up the same native APIs that we use internally to build all our iPhone applications."

iPhone Roadmap Event

 

Sorry, could not resist... But stay tuned, we'll return with some actual IxD relevant topics shortly!

iPhone App Store turnover

I don't believe it for a second when Apples suggests that their 30% cut merely will cover their costs. There is money to be made from the App Store, both for Apple and for the developers.

How much does people spend on mobile content?

Example: Norway is a small country in northern Europe with 4.5 million people. There is 4 million mobile subscribers in Norway, about the same as the number of iPhones sold. The total market for mobile content is about $200M. (These numbers are a couple of years old and were growing.) So on average every subscriber purchased mobile content for $50 yearly. The figure is the total turnover on the two available billing mechanisms, "Premium SMS" and "wap-billing" for all Norwegian operators. The figure is gross payout (including sales tax) from the operators to the content providers. Mobile content billed otherwise is not included but is probably negligible.

The revenue was divided in these categories:

• $20M Directory services (people looking up phone numbers via SMS) 
• $30M TV-interactivity (typically voting by SMS for televised competitions and reality shows) 
• $30M Adult 
• $80M Phone Content (40% ringtones, 40% games and 20% other content)

(The last couple of years ringtones have lost share while games and other has increased.)

Average spending was $50 per year, but that does not mean that every subscriber spends the same amount. For example, the age group 20-29 spends twice as much as the average. In fact, most people don't buy content at all. I don't have exact numbers, but our educated guess was that around 15% of the subscribers did ALL the shopping. That of course makes each buying customer even more valuable.

So, are iPhone owners "big content spenders"?

You can argue that the iPhone is a luxury item sold in fashion stores to people that just want to flash their elite status. Or you may argue that the iPhone is every geek's dream come true, and that that they can't wait to stuff their phone full of fun, useful, bizarre and/or interesting applications. Apple probably knows, but I don't, so I won't make a guess. No, I won't. No.

Ok, ok: I think iPhone owners will purchase more music, video and applications than other phone owners. The reasons I believe this are many, probably the subject for a separate blog post. Basically, the demographics align. And shopping phone content on most operator portals is a deeply broken experience: not receiving what you pay for, unethical business practices (like inadvertently being signed up for a subscription that you can't cancel without changing phone number) etc, etc, etc. Compare with iTunes. Apple knows how to move goods smoothly and quickly past the checkout counter without hassle or unwanted surprises.

 

App Shop revenue

Of the above $50 per user mobile content revenue, only "Phone Content" would be sold through the App Store. So let's say $22 is a reasonable revenue per iPhone owner. (Compare this to music sales on iTunes. Each iPod owner buys $15-$30 worth of music per year on iTunes.) On what will the $22 be spent? Mainly on games.

By the end of this year, if Apple has sold 10 million iPhones, this would mean a $220M turnover and a $66M contribution for "running the store". By next year end, if Apple has sold 25 million iPhones (analysts prediction) Apple would receive $165M for running the store.

Man, this was easy! I could become an financial analyst. He-he. Maybe not. If you think Norwegians in general are very different from iPhone owners in general, adjust the numbers to your liking. But I believe a $22 per user turnover is likely for the iPhone App Store.

When the last optimist turn pessimist

Michael Mace wrote an excellent post named "Mobile Applications, RIP" on his blog. You have probably read it if you "follow the mobile space". A lot of people has added their points, I'll add some links at the end of this post.

 

The end is nigh!

The gist of Michael Maces post is: Downloadable mobile applications is dead, crushed by a fragmented market and restrictive business practices. The problems are now so bad that even the mobile web looks like a better way to deliver new functionality to mobiles.

Much of the discussion following the post has focused on the choice of technology and not on what I believe Michael Mace really is pointing to, the broken business model. Selling standalone mobile apps to end users does not work. Hardware bundling is gone, web storefronts are gone and the carriers don't care about you. You are left with Handango who wants 50-70% of your revenue.

The web is supposed to save you, but the troublesome part is: What are you going to do between now and 2010 or so when AJAX capable browsers and browser APIs have reached mass market penetration?

All may not be lost. Before you put on a robe and join the choir chanting: "The next big platform will surely save us all", check out this emerging trend:

 

Manufacturer storefronts

Handago and scattered web storefronts is out. Carrier portals is out. Software will be sold through on-device + web storefronts operated by the manufacturers.

• Software for the iPhone (and iPod touch) will be sold through iTunes. 
• Software for Nokia devices will be sold through Ovi. 
• Microsoft has just bought Danger who runs a an application storefront for Danger devices.

Manufacturers will act as a gatekeepers, deciding which are worthy of release, and publishing only approved applications.

"So what", you may ask. "How is this different from the operators controlling/selling applications?" Well, you might end up on the short end of the stick again of course, but there is a couple of pretty significant differences from operator portals:

 

Per device instead of per operator

The operator will ask you to support all devices they have sold over the last 2-3 years regardless of whether they align with your target audience or not. You are forced to create a lowest-common-denominator solution, when instead you probably want to create an amazing service targeted to music phones, phones with GPS or whatever.

Kristian Segerstråle (Glu) to MocoNews:

“It’s time to go from defining the product based on the carriers’ desires to doing something completely new.”

“Who says you have to support 1,000 handsets from the start? How about 10 and see what the uptake is?”

Besides, forcing developers to support sub-standard devices is unhealthy for the mobile ecosystem. These devices needs to die.

 

Marketing

One of the main problems with the operator portals was that there is no way for the consumer to separate the wheat from the chaff. Operators treat everything like another SKU because this kind of content is low priority. Depending on the operator, you may not even be allowed to publish your company name. And either you pay the operator out of your nose to get exposure or you get zero exposure (my experience, anyway). 

Developers on the other hand want to build their brand and the manufacturer storefront might be better than the operator portal. At least it can't be worse.

 

Meanwhile, back on earth...

In contrast to the highly speculative prose above, here are a couple of statements from people developing mobile applications and services right now:
Barbara Ballard: "This quarter is not particularly different from other quarters: we get far more work designing applications than designing web sites."
Simon Judge: "From my angle, there has never been a time when there are more companies looking for someone to create their native application."

And from where I sit, I see mobile applications being developed like never before. It used to be garage operations that made mobile apps, now large companies are entering in a big way.
We see a lot of activity from ISVs that build mobile "companions" to their desktop-based software.

A couple of years ago everything was strictly vertical (and ran on Windows Mobile only). Now we see mainly horizontal applications like CRM systems, project management etc. for feature and smartphones in general. The clients sit in meetings waving their jailbreaked iPhones and demanding their software be "as easy to use as the iPhone". Its a joy to be an Interaction designer these days!!

The mobile applications I'm involved with are typically sold directly to enterprise customers (not end-users), installed and maintained by the IT department and the business model is the same as desktop software. A hefty per-user fee.

 

More on the conversation here:

Michael Mace: Mobile applications, RIP
Mike Rowehl: Native Mobile Apps vs Mobile Web Apps
David Beers: Have mobile apps had their moment?
Rafe Blandford: The future of mobile development?
Dean Bubley: Standalone Mobile Apps vs Web Apps on Mobile

Flat Music Player version 2

 

 

Click the image to see the animation (It opens in a new window).

Please note that the text below is necessary for understanding the animated example above. Well, I think so anyway :-)

The content itself is the interface

I've been thinking a bit about "content is the interface" lately. The idea is to avoid deep hierarchies of thin information by intensifying the information resolution on the screen. Edward Tufte is the great proponent of this idea and lately it has been made current by the MEX conference as their Manifesto #1: Content itself will be the interface of the future. 

It is easy to see how to employ this paradigm when the information is manageable, when it's on a "human scale" so to speak. The in-flight entertainment selection in an airline seat, the variety of coffees in a coffee shop, the selection of cars at a dealership. You can take in the information and hold it all in your head at once. (Deciding what to choose is of course a different matter…) 

It is certainly both possible and advantageous to use the content itself for most navigation and interaction tasks. But when I tried to apply "content is the interface" to music there were a few additional issues. A typical device can store 8GB of music; that is 2000 songs or about 250 full albums. Many devices can store a lot more. Not to mention the millions of songs you can browse when you want to buy music. Presenting all this music directly is not possible. 

In addition, music is sound; you can't actually see it or touch it. You have to choose a visual representation and I've used album art because that’s an abstraction that most people understand and are used to. With digital distribution, there is some movement away from albums in favor of individual songs. I've considered using images of the artists as visual representation but it does not work well. Album art has a strong visual identity and is associated with a particular selection of songs. Is scales well; you are able to make out the album even when scaled way down. This is not the case for photos of artists or bands. They change randomly over time and they don't scale well. 

Spatial reasoning

In addition to this, I wanted to try to take advantage of spatial reasoning and spatial memory to make it easier to find and navigate stuff. Let the user see the scope of information available. Start by showing  the big picture. When it makes sense, let it behave more like real-world objects. You can normally pick up objects where you left them off. They don’t move when you are not watching, something digital objects often do. (Insert your favorite joke about spouse here.) Over the last couple hundred thousand years our brains has developed a fantastic ability to take in and store where stuff is in our immediate surroundings. Since mobile screens are a part of our immediate surroundings, we should try to take advantage of this ability. It might sometimes make user interfaces a bit less confusing. 

This is version 2 of the concept player that I fiddled with over at OTA. This time I've used a larger touch screen (the size is equal to the upcoming Nokia Tube). 

Current music players

 

Screens from a Nokia Series 40 phone and the Sony Ericsson 960W phone. 

The problem with the illustrated players is that the vast majority of the screen is décor and "computer administrative debris". The information value is extremely small. All that fits on the 240 x 320 pixel screens are three band names (Nokia) or four song names (Sony Ericsson). 

Nearly all players use a hierarchical presentation where the user steps through levels; Genre, Artist, Album, Song. The icons are genetic and do not add any information value. The layout of the W960 music player only uses half the screen width for the song name. The result is that most song names have to be truncated even if they are rendered in a small font. 

Playlists

Terminology problem alert: Playlists are normally used to describe a (small) selection of songs on your device. For lack of a better word, I also use playlist to mean the collection of all songs on your device including their metadata. 

Going forward I expect that playlists may become more important than the music files themselves. Renting (subscription) may well be the way to consume music in the future, and the list of your music will be more important than the files themselves. A development similar to how bookmarks has moved away from individual browsers and computers and up in the "cloud" from where they are accessible regardless of device. Obvious example: del.icio.us. If you change your music subscription from, say, Rhapsody to Napster, you just take along your playlist and continue enjoying your music. 

I don’t think there should be any significant difference between a streaming player and a file player. If you add an album, it should appear in your music collection the same way as empeethree files does. ("Internet radio" streaming is different of course, as you don’t select music, you select a station.) For performance reasons subscribed music probably have to be stored or cached locally. 

Hardware

These design sketches are made with the nascent/emerging trend of high resolution widescreen touch phones in mind. Examples are Sony Ericsson Xperia, Nokia Tube and Apple iPhone. 

 

Note that the sketches focuses on the interaction model, not on the feature set or graphic design. Also, the intention has been to work on something that might be useful on actual emerging hardware, so no Jetson style 3D holographic interaction models at this time. Most of the sketches are so called Hi-Fi sketches because it's easier to convey the ideas with something that resembles a product than with plain wireframes. 

The basic structure of the app

 

• The layer named “Albums” holds the music library. It is divided into rectangular sections (labeled Indie, R&B etc…) 
• "Hype" is a closable panel. It has an, um, "river" where info related to the music floats by slowly. It has a drop area where you can drag and drop content that you want to subscribe to. (No, let's not call it the shore or the riverbank :-)) 
• "Shop" is another closable panel that contains a music shop.

Albums (the library)

 

The busy impression of this well filled music library is intentional. The library contains about 200 albums. (And movies. I'll stick to music to avoid distractions, but I don't see any reason a movie player needs to be distinct from a music player.) 

• No particular distinction is made between music that are streamed and music that are files. 
• The albums are not sorted alphabetically. New and recently played albums appear on the top of each pile. The reasoning behind this is that the human brain understands space well. We know approximately where things are when they have a position in a physical space. If you want play a song you tend to remember that it was "somewhere in the middle of the pile over there". 
• Albums that are played often grow bigger in size. 
• Classification (Rock, Indie, R&B etc.) is based on metadata for the music. The user can change classification to fit personal preference. Missing album art is automatically retrieved from relevant web services. It is also possible to identify unknown music tracks via something like Gracenote. 
• Suggestions are based on playing patterns and/or suggestion engines like last.fm, Pandora and social engines "people you know listens to…".

The bottom part of the screen displays a bar when music is playing. It holds player and volume controls. If the hardware has separate player and volume controls, that's good. That will make it possible to control the player when in the pocket. 

The top part of the screen needs a back button and probably a search function (not shown). 

 

Clicking on a category zooms/expands the category to fill the screen. I have show the albums organized in a grid in this example. But keeping the position and size-difference when zoomed-out needs to be investigated.

Clicking on an album opens it and starts playing.

Individual songs 

 

The player needs a mechanism to access individual songs. Tapping the album above the play-control in the lower left corner could pop up a more detailed player.

Discovery

The closable panel labeled "Hype" is for exploring. Discovery of new artists and bands is an important function. 

• Contains related hype (blog posts, news, etc) about the artist or band. 
• Can be accessed while listening. 
• Displays information related to the currently selected music or to music in general if no particular album is selected.

• The hype blurbs or "story boxes" flows down from the top. 
• You can subscribe to any of the sources you find there by dragging them over to the leftmost column. 
• I've used last.fm to illustrate one source of info, but sources can be anything, HTML-based, API-based, RSS based, etc.

Accessing hype content 

 

Tapping any story box in the Hype panel zooms to the full content. 

Shop

 

The Shop should display its content in a nice and tidy fashion, as shops do. Since it's so tidy, gridlines between sections don't seem necessary. Navigation should probably be a combination of Tap-to-zoom  and Drag-to-scroll. I haven't spent a lot of time on the shop to be honest. 

End notes

The Flat Music Player version 2 ended up having only play/pause controls, a back-button, and two closable panels that are manipulated by gestures. An actual product would need a little more; it has to be possible to type in names for sections, probably URLs, and ways to connect to friends. But I think this shows that you can get quite far using the content itself as the interface. 

I was looking for more plasticity and viscosity than I could wring out of Flash, hopefully the animation gives some sense of the intent.

[UPDATE] I entered the The Flat Music Player in the MEX Design Competition. If you like it, you can vote for it.

Inspiration

Lumines
Amaztype
last.fm
Hype Machine
Universe by Jonathan Harris
NokiaDesign

More on mobile screen size trends

I recently wrote about screen size trends and the trend is clear: 240 x 320 (aka QVGA) is the new baseline screen size, both for feature phones and smartphones. Smaller screen sizes like 176 x 220 is disappearing and the 128 x 160 size is pushed down into basic phones.

QVGA will be the new small size, but it is not clear what the new dominant large screen size will be, or if there is going to be any.

What is pretty clear is the trend towards widescreen and landscape orientation for high-end phones. QVGA and larger phones are getting automatic screen rotation. Landscape is often preferred for media consumption and handheld movie consumption has been "promising" for quite some time.

The primary design objective for the iPhone (in my opinion) was to make the optimal video iPod. When you have 70% marked share you need to grow into new markets. Several new phones from other manufacturers seem to have the similar design objectives.

I have compared some available and coming devices that could represent a trend. LG and Samsung seems to be partial to the 400 x 240 screen size. They have released several devices with this screen size the last year or so. Nokia has recently tipped their hand with the Nokia Tube. The name obviously refers to a video use-case. At MWC in Barcelona in February they showed a demo that ran on a 360 x 640 screen. This gives a 16:9 resolution, the same as HDTV. 640 x 360 is also called QHD for "Quarter HD".

The Sony Ericsson Xperia has been developed for the US market specifically. It runs WinMob 6.1 on a 800 x 600 screen, a resolution not supported in any official Microsoft documentation, but I guess Microsoft saw fit to go the extra mile in this case :-) The Xperia has an extreme resolution, it has more pixels on 3 inches that a good ole VGA desktop screen. I'm not sure that the Xperia exact screen size and resolution represents a trend, is kind of an extreme case. But it underscores the general high-end widescreen trend.

Pixel dimensions and aspect ratios

I used 9 as lowest common denominator for ease of comparison.

Physical screen size

Most of the devices uses 3–3.5 inch screens, something that does not leave much space for a keyboard. The phones are either clamshell or pure touch. They are also fairly similar in physical size.

Gestures, pixels and bandwidth

After years of designing for really small screens and always in portrait orientation, these devices sure feels like drowning in pixels. I absolutely expect to see some great and innovative user interfaces coming out of gestures, pixels and bandwidth!

[Note:] No mention of Motorola here, for the simple reason that I have no idea what they are up to.

[Edit] Just to be clear, the reason I'm exited is because the phones above makes it possible to design for a wide screen. I'm aware that many or most of the standard functions in most of the phones above is intended for vertical use.

Mobile screen size trends

Over at mBricks my colleagues have put a lot of work into the device database (we work with WURFL data and contribute back as well as we can). I took the opportunity to take a closer look at screen size trends. The data I have covers about 400 different device models sold from 2005 to today.

This shows the the most significant screen sizes, from the smallest to the largest. I have added a couple of upcoming phones as well, they are the ones with the dotted lines.

Over the years the relative screen size difference has increased. The difference between the smallest (128 x 128) and the largest (800 x 480) is now a factor of 23. That means the largest screen is 23 times bigger than the smallest one.

You can see that the smaller screens have a portrait orientation and the large screens have a landscape orientation. Between them are the phones that can change orientation, they can work in both landscape and portrait. 240 x 320 is the dominant screen size overall.

Resolution

I did a rough dpi (ppi to be exact) calculation for some popular phone models. The pixel density actually increases when the pixel count increases. The screens are not only getting bigger, they are getting sharper at the same time.

There is an upper limit to what dpi is meaningful. At a certain density, the eye can no longer see any difference. If the specs are correct, the upcoming Sony Ericsson Xperia X1 will have a pixel density of 298. That is the highest I've seen on a mobile phone yet. The human eye can resolve about 340 dpi at one foot viewing distance IIRC, but tests show that most people don't see much difference between a 150 and a 300 dpi image. So 298 dpi should be plenty.

Unfortunately, for LCD screens, increased pixel density doesn't give us more brightness. More brightness makes the screen easier to read outdoors and is more important than resolution from a usability perspective. OLED displays will help with this, but we are drifting off topic.

 

Yes, a grand total of 26 different screen sizes. I only counted phones that had a color screen, ran Java and had a browser.

As you can see, just a handful of variants makes up the majority of phones. Lets take a look at them:

 

It is obvious that 240 x 320 (also called QVGA) is on a roll. It is by far the most common and it is growing rapidly. If you develop, this should be your target screen size.

 

When you develop, you primarily need to worry about the width of the screen. For most practical purposes, the height takes care of itself. I have lumped together 176 x 208 and 176 x 220 for this graph. There is still a lot of them in the market, but their numbers have been decreasing since January 2007. In 18 months this screen size may be out of the market. Nokia haven't launched a phone with this screen size in 1,5 years. I have also lumped together 128 x 128 and 128 x 160.

Phone screen sizes has had a tendency to come in pairs. Each manufacturer had their own variation on large-screen high-end models and small-screen low-end" models.

Manufacturer	small screen	big screen
Sony Ericsson	128 x 160	176 x 220
Nokia	128 x 128	176 x 208
Samsung	128 x 160	176 x 220
Siemens	130 x 130	132 x 176

 

Individual variations is disappearing, the trend is:

Manufacturer	small screen	big screen
Everybody+dog:	240 x 320	?

 

The majority of odd-sized screens are disappearing. No phones with the above screen sizes have been introduced for at least 4 quarters.

The dead and dying are partly BenQ-Siemens (who has left the market), partly the old Sony Ericsson P-series and partly the Nokia hires versions: E60/70 and N80/90.

The odd ones

What about the other ones, the not so popular, but not dying? These are the ones that do you in.

Fashion phones:
128x220	Samsung F210
240x400	LG favorite. Prada and Viewty.
Handheld touchscreens of the iPhone variety:
240x440	Various Palm and HP
240x480	LG KF700
Typical Palm/Blackberry form factor. US enterprise with portrait or square displays:
240x240	Samsung F210
240x260	Blackberry Pearl 8100
320x240	Various
320x320	Palm and Samsung
Clamshell:
640x480	HTC X7500, Qtek 9000, etc.
800x352	Nokia E90 Communicator
800x400	Sony Ericsson Xperia X1

 

Future

What will be the dominant screen sizes in the future? Well, the 128 width screen sizes are moving down from feature phones to basic phones and there are very few manufacturers that still uses 176 width screens. It looks like 240 x 320 is the new base line.

What will the dominant large screen size be? Let's hope there will be one. Right now there are a lot of different ones, every manufacturer has their own and I can't see a clear trend. The two hottest form factors right now is the handheld touch screens (ala iPhone) and the QWERTY clamshell (ala HTC).

For the handhelds, the 320 x 480 iPhone size is a possibility. It has an ok resolution for a 3,5 inch size. The Nokia "Tube" phone will have a 360 x 640 size (my estimate) and is another possibility.

For the clamshell form factor, 640 x 480 screen size would be my bet.

About the data

The data includes all the 400 different phone models sold in the Norwegian market from 2005 to 2008. I've given them four quarters of life time past the quarter they were introduced. That means an average of 14,5 months. This might be a little short.

I have left out basic phones. Only phones that has a color screen, support for java and a browser are included.

The Norwegian market is not very different from most other European markets. It's GSM only, the majority of handsets are sold with 12 month operator binding. Sony Ericsson, Nokia and Samsung are market leaders. Motorola is weak, HTC is up and coming in the enterprise market. The data includes the iPhone even though it has no official distribution. There is quite a bit of "black" iPhone imports.

Reliable sales figures for each individual device is not available, so the data is not weighted on popularity. But I don't think popularity would give a very different conclusion. I've checked "top ten" lists where available and none of the "odd" screen sizes seem to be big sellers.

Note: all sizes are written as width x height. So a 240 x 320 screen is portrait while a 320 x 240 screen is landscape.

Flat Music Player

I'm at the two day Over The Air conference in London. It's a combined code camp and conference. As is customary, there is a hackathon going on. Yesterday I was sitting in a couple of sessions bored (not all of the sessions are good to be frank) and I thought I'd join the competition and submit a design. I don't know if something that doesn't run code is admissible at a code camp... we'll see.

Here is the submission:

The problem

The majority of current music applications on mobile phones are surprisingly poor.

    

Screens from a Nokia Series 40 phone and the Sony Ericsson 960W phone.

The problem with the illustrated players is that the vast majority of the screen is décor and "computer administrative debris". The information value is extremely small. All that fits on the 240 x 320 pixel screens are three band names (Nokia) or four song names (SE).

Nearly all players use a hierarchical presentation where the user steps through levels; Genre, Artist, Album, Song.

The icons are genetic and do not add any information value. The layout of the W960 music player only uses half the screen width for the song name. The result is that most song names have to be truncated even if they are rendered in a small font.

Flat Music Player

This is an attempt to apply a different approach than the hierarchical lists used in music players in most current phones. 

The assumption is that people do not think about music in hierarchical manner. People organize their music according to their own head, primarily in a spacial image.

I don’t think there should be any significant difference between a streaming player and a file player. If you add an album, it should appear in your music collection the same way as empeethree files does. ("internet radio" streaming is different of course, as you don’t select music, you select a station.)

I wanted to design this so it could theoretically be implemented for current phones on the market today. That means 240 x 320 screen size, both touch and softkeys and a layout that can be adopted for both portrait and landscape.

I’ve tried the following:

• Use the content itself as the interface
• Flatten the information space somewhat
• Provide for spatial reasoning and spatial memory

Main view

The job of the main view is to give you a complete overview. It is based on album art because album art is (currently) the best visual representation of the music. At this zoomed-out scale, the albums themselves are tiny color blurbs, but you can still recognize the individual albums.

The albums are not sorted alphabetically. New albums appear on the top of each section. The reasoning behind this is that the human brain is good with space. We know approximately where things are in space. If you want play a song you tend to remember that it was "somewhere in the middle of the Indie section".

A search is started merely by typing from the keypad. Search should be modeless, i.e. searching artists, album titles and songs at the same time. It might even get fancy and return music from the sixties if you search for "1960".

Classification is done automatically. The user can change or override classification.

Suggestions are generated automatically.

 

The sketch is charcoal, or shades of gray (standard designer issue), but it must be possible for the user to add and change around colors. MySpace-ification is admissible :-). The size and layout of the sections should be user-adjustable.

Zooming interface

Each section is selectable. When clicked or tapped on, the section expands or "zooms" to fill the entire screen. 

In this view, the albums can be scrolled and individual albums can be selected.

I made the text too small unfortunately, it has to be bigger, but must fit the same grid.

Playing

Clicking on an album opens the album, displays the song list and first song is ready to play.

The progress bar is shown under the song name, so you see what the next or previous songs are. 

This view should also offer means to discover other music "like this", bands that have influenced this band, followers, etc, etc.

The play/pause player controls are mapped to the joystick push. Fast forward and rewind is mapped to the joystick right and left. The concept of "Stop" has no use on a digital player. 

Sharing

I didn’t have time to do a "hifi" mockup of this screen.

The idea is that the recipients on top are "near", for example Bluetooth devices nearby. The recipients below are "further away", they are IM friends, Facebook, Address Book contacts etc.

Browsing new music

      

Straightforward horizontal scrolling interface. You can switch between a song list and and band/album information. 

It turned out looking like it’s more or less lifted from iTunes, I’m sorry.

Needs work, but time is up. I have to submit now.

[Update]

Lots of cool demos here. I especially liked the Social Network Open Butler (presented by Robert "Jamie" Munro). It aggregates data from several social networks into your mobile phone address book. It can add a picture found on Facebook and a place from, well, from somewhere else :-) I'll link to it if I can find something posted later.

Phone Phight by Russ Anderson is a game in the starwars genre where two people battle it out using their Nokia N95s as "lightsabres". The phones are bluetooth connected and uses the game uses the accelerometer and bluetooth to decide who wins. Hopefully someone caught it on video!

[Update]
Here is a video showing Phone Phight:

Administrative debris

It occurred to me today that this blog is one year old. Hey, it's an anniversary! Actually it was a couple of days ago, but anyway.
I guess I'd like to use this opportunity to add a bit of administrative debris to this blog.

The thing that prompted me to start the blog a year ago was a lack of resources on Interaction Design applied to Mobile. My plan, if I had one, was to write down my thoughts about Interaction Design in general and Mobile Interaction Design in particular. I mean, If I can't read what I write, how can I know what I think? That is not my saying, it sounds like a Mark Twain quote, I'm not sure. But I think it's very appropriate for me. I spend some of my time deconstructing user interfaces, trying to reverse engineer what thinking (if any) that went into a particular design choice. From time to time I see really brilliant work, but most of the time investigating UIs mainly provides clues to why the design fails. And understanding failures is as educating as understanding successes. Writing helps me organize myself. If it can benefit anyone else, it makes me very happy.

Moving to London

Come June we are moving to London. "We" means my wife, me and our toddler. I'm really excited about this! There is a lot of mobile innovation and development going on in London and I look forward to being a part of that.

The aviator is not me

From time to time I meet people that I can't recall having met before and they may say things like "I didn't recognize you with the beard". I think it might be the aviator in the blog banner. The aviator is metaphorical. I should probably update the About page. If you want to know what I look like check the buddy icon on for example flickr.com/mhjerde.

The future

I plan to write more about hardcore mobile IxD issues. Apart from that I don't have any great changes planned: There is not going to be any product reviews, if I write about a particular product, its usually from a IxD angle. There is no regurgitated news here. I may comment on industry news/events tho. I probably post 2-3 times a month. If you like the blog, the good news is that it's not going to change much. If you don't like it, the bad new is that it's not going to improve much either.

On that note, my advise is: Don't read too many blogs. Spending time sketching, thinking and talking to your users is the best use of your time. If you need to organize your thoughts, write. I can't offer you any advice on writing, but if I recall correctly Mark Twain has one: Write drunk, edit sober. He-he, might work. Just don't skip the last part. Get your ideas down on paper, into Photoshop, Illustrator, Eclipse, Visio, whatever your tools are. Nothing gets realized as long as it is only inside your head. If reading blogs and news takes away from that, unsubscribe today, including this blog.

See you.
Morten