This article will explain the fundamental concepts for utilizing smart device technologies and sensor data in order to understand context and introduce “adaptive thinking” in to the UX professional’s toolset. I will demonstrate the importance of context when designing adaptive experiences, give ideas on how to design adaptive systems, and perhaps inspire designers to consider how smart devices and context aware applications can enhance the user experience with adaptivity.

Examples of Adaptive Systems

An early example of an adaptive feature is found in GPS navigational devices. Using the device, a user is able to easily locate and navigate to any location he can drive to. When the sun sets or while driving through a tunnel, the system automatically changes the interface color to a dark “night mode” so not to blind the driver with a bright light from the device. The system knows the user’s exact location and the position of the sun, and by understanding these two factors, the system maintains a safe driving environment by adapting to the user’s needs.

GARMIN Zumo 660 Day and Night Interface

Adaptive design is about listening to the environment and learning user patterns. Combining smart device sensor data, network connectivity and analyzing user behavior is the secret sauce behind creating an adaptive experience. By combining these capabilities, we not only understand the context of use; we can also anticipate what the user needs at a particular moment. Google Now is an interesting example of an adaptive application that gives users answer to the questions they’ve thought rather than typed. Through a series of smart cards that appear throughout the day on the user’s mobile phone, Google Now tells you today’s weather before you start your day, how much traffic to expect before you leave for work, when the next train will arrive as you’re standing on the platform or your favorite team’s score while they’re playing. It does this by recording and analyzing your preferences while using your phone. For example, updates on your favorite sports team are based on your web browsing and search history. Another example is by analyzing your current location, previous locations, and web history, Google Now presents a card with traffic conditions on route to your next likely destination.

As UX professionals, we understand that mobile users are not keen to use the virtual keyboard and we try to avoid the necessity as much as possible. By utilizing the user’s personal behavior as a sensor together with smart device capabilities and enabling voice commands (similar to iOS’s Siri), Google Now creates an adaptive experience that helps users avoid using the virtual keyboard, thus further adapting to the mobile user’s needs and helping users quickly get the information they require on the go.

Adaptive systems are not only limited to mobile devices. Ubiquitous computing (UbiComp) is a surround of smart devices and networked digital objects that are carefully tuned to offer us unobtrusive assistance as we navigate through our work and personal lives.  Similarly, ambient intelligence (AmI) refers to digital environments that are sensitive and responsive to the presence of people.

Nest, The Learning Thermostat

Nest, The Learning Thermostat, is a great example of an adaptive system integrated to home environments.  Using a variety of sensors for temperature, humidity, touch, near-field activity, far-field activity, and even ambient light, it can detect whether there are people home and how active the home is at any time. Doing so, it can automatically cut up to 20% off a home’s heating and cooling bills.

When no one is around, Nest learns to turn the heat down. When you come home from work, it knows that the heat should go back up. After the first few weeks, it learns when you come home from work and can turn the heat up before you arrive so that you come home to a warm house.

In 1991 Mark Weiser wrote,

“The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.”

Nest is a great example of UbiComp and how technology can disappear into our surroundings until only the user interface remains perceivable by users.

These devices create contexts of sensor and user data that provide a superior user experience by anticipating what the user might need before the need is expressed, and this is the future of UX design.

Adaptive Thinking

In contrast to traditional desktop systems, mobile devices are normally used in many different situations. However, mobile applications nowadays do not often utilize the context of their use and hence are only usable for very specific purposes. For example, a city maps application for local businesses is used in different contexts, walking through town, at home, or used with no network connectivity. Today’s users can customize their device’s system through modifying their preferences and settings or by choosing what application works best for their needs. Even after the implementation of user-centered design processes that assures a certain degree of user acceptance and yields a richer understanding of the context, it is impossible to anticipate the requirements of all users and map it to a single best or optimal system configuration.

“Adaptive thinking” is a mindset that provides the tools necessary to significantly improve the user experience and enhance the intended purpose of the product by utilizing the technology that is readily available in every pocket. It is about learning the environment and the user and adapting to his current needs and situation. Therefore, designers should first design for the context of use and then design the set of functions that are triggered in relevant situations.

Here is an instructive case where adaptive thinking was used to create a mobile app for a bike sharing program. A bicycle sharing system, also known as bike rental, is becoming an integral part of major cities around the world. Bicycle sharing helps reduce traffic congestion, air pollution and encourages locals to maintain a healthy lifestyle. A user looking to rent a bike will turn to a mobile application to look for the nearest bike rental station with available bikes to rent. The user will then use the application to navigate to the rental station (if he is unfamiliar with the city); this is the core functionality of the application. An adaptive system will realize when the user has arrived at the bike rental station and automatically offer additional options, i.e., adapt to the current situation. For example, it may offer him a quick way to rent a bike, a feature that was not available before arriving at the rental station. During the rental period, the system will anticipate the user’s needs and offer nearby bike rental stations with available parking spots and show him the current balance for the rent time.

By using the assisted GPS device capabilities, network connectivity, and by understanding the user’s story at any given time through the product lifecycle, adaptive design will provide users of the mobile application a reliable extension to the bike rental program.

Adaptive and Responsive Design

An adaptive system is one that adapts automatically to its users according to changing conditions. Responsive design is a subset of adaptive design, an approach to web design in which a site is crafted to provide an optimal viewing experience across a wide range of devices. In my UX magazine article “The Multiscreen Ecosystem” I discuss how responsive design can also be adaptive by understanding the context of using a mobile device and by designing contextual paths.

Context for Adaptability

I quote below from the 2007 book The Adaptive Web, which talks about the importance of context for adaptive Mobile Guides: It explains adaptivity in the scope of mobile systems as context-aware computing, i.e., the ability to use information in the current context to adapt the user interaction and the presentation of information to the current situations of users.

“Understanding the context is an important prerequisite for the adaption process. Context is not just the location, but encompasses also information like the ambient noise or lighting level, the network connectivity or bandwidth, and even the social circumstances of the user. Furthermore, systems have to anticipate the user’s goals and intentions, which might be inferred from their actions or from physiological sensors and appropriate environmental sensors (e.g. light, pressure and noise sensors).

One prerequisite for adaptive systems is the proper assessment of the user’s situation. For this purpose, systems need to rely on a representation of relevant situations. Depending on the supported task, situations can be characterized by many different attributes. Therefore, designers of suitable adaptation for mobile devices need to look at a variety of spatial, temporal, physical and activity related attributes to provide effective assistance. For example, a mobile  application that assists users in a shop needs to know about the current spatial environment of the users (e.g. which products are nearby), the temporal constraints of the user (e.g. how much time is available for shopping), the general interests of the users and their preferences (e.g. if the user prefers red or white wine with tuna), details about the shopping task itself (e.g. which items are on the shopping list and for which purpose the products are needed) and maybe even about the physiological and the emotional state of users (e.g. whether users are enjoying the shopping or not).”

That being said, understanding the locational context and the user story is now easier than ever before. We can utilize the fact that we carry our phone wherever we go. This phone is packed with technology and information about the user that designers can use to understand context. The highly sophisticated, advanced technology in our pockets not only allows designers to analyze if the user is walking, standing, or in a loud or quiet environment, but also can help us understand the precise location of a person within a department store or even a specific aisle.

AislePhone, an Israeli startup currently in the beta stage, is developing a platform for precise in-store positioning that can understand the exact position of a person within a specific aisle level. With this technology shopping with your mobile phone at hand will be a common experience, as mobile apps for supermarkets and other large retail stores will utilize locational and user data to enhance the shopping experience, much like a personal shopping assistant in your pocket.

Google Indoor Maps allows users to view and navigate floor plans of several kinds of commercial locations such as airports, department stores or malls within Google Maps.

This technology not only understands your indoor location, but on what floor you’re on. Depending on the data available, the map will show notable places in the building you’re currently viewing, including stores, restrooms or the nearest foot court.

With this type of technology, “you are here” directory maps will become obsolete when you enter mall or a department store. You will be able to understand your location and orient yourself using a smartphone, and this experience will adapt to your specific needs. For example, apps will offer you relevant discounts as you walk through the mall or highlight several shops based on your gender and age.

Designing Adaptive Systems

Adaptive design integrates both subtle and obvious features. Often, adaptive qualities can be very subtle and unobtrusive: sometimes a seemingly small adaptive feature can greatly improve the overall experience. For example, did you ever notice that Google Search can read your mind?  When you start typing, Google autocomplete (Google Instant) knows what you’re thinking even when you enter only three letters searching.  It does this because Google Search considers and records all search queries within a session in order to have a better understanding of the user’s intent.  When a user searches for “The Beatles,” Google understands this as part of a research session and will help you quickly discover Ringo Starr or Paul McCartney as you enter the first three letters of their name; it understands the context of your search and compares it with other similar popular relevant results. Another example of a subtle feature that helps enhance the user experience could be a system for students that adjust the difficulty of test questions according to prior answers. Or a music discovery app that looks into your current play list and adapts to your taste, helping you discover additional music you may like.

Google Instant understands the context of your search

Although the experience should always be unobtrusive, adaptive interfaces need to be obvious so users understand the context for the adaptation and always feel in control.  For a better experience, applications should allow users to manage adaptive features. For example, if at nighttime the interface changes to a darker night mode (like in navigational devices), the user should always be able to change it back manually. Or, if entering a shopping mall triggers a different experience, the user must understand the context for this adaptivity and want to embrace the added functionality.

Charles Darwin wrote,

“It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change”.

In today’s world, a person carries in one pocket more advanced technologies than ever before possible. An intelligent device like a smartphone is embedded with highly sophisticated sensors.  These sensors, together with advanced computing power and network connectivity, can help us analyze and understand the context of use. The smart device’s ability to analyze the context of use in real time, together with understanding the user’s story, allows opportunities to provide an even greater user experience by adapting to the user needs.

Analyzing User Behavior

Similar to the Google Now example, analyzing user behavior and his interaction with the digital world can yield us a great understanding of the user’s context. Analyzing the user’s search patterns or what applications he downloads can tell us about his preferences and hobbies. Tracking current location and location history can yield us the user’s surroundings and the physical boundaries of his life, so we can understand what subway station he takes to work, or where he likes to eat his lunch. Note that when this is done without the knowledge of users, it may be considered a breach of browser security and illegal in many countries.

Here is a practical example of how analyzing the user’s behavior could help in creating an adaptive system. In the now famous Google Glasses video, we follow the user throughout his morning as he eats his breakfast and then leaves his house heading for it’s the subway. Upon arriving at the subway, the user receives a message that subway service is suspended and is offered a walking route. As useful as this may be, a true adaptive system will analyze the user behavior as the user gets up and will warn the user ahead of time that the subway service is suspended.

Understanding the user behavior (whether he takes a subway or walks to work) and connecting it with available information online allows us to understand and adapt to his needs.  Most times, using one data source is not enough, combining the technologies (network connectivity, user behavior, and sensor data) is the only way to understand context. For example, we can gauge the outside temperature by using the user’s current location combined with online weather information, and with this data offer numbers for nearby cab companies in addition to a walking route, assuming he may not wish to walk to work in the rain.

Making Use of the User’s Story

Behavioral targeting or personalization refers to a range of technologies used by online website publishers and advertisers which allows them to increase the effectiveness of their campaigns by capturing data generated from website and landing page visitors and adapting to their needs.  Personalization technology enables the dynamic insertion, customization or suggestion of content in any format that is relevant to the individual user, based both on the user’s explicitly provided details and his implicit behavior and preferences.

Another aspect of personalization is the increasing prevalence of open data on the Web. Many companies make their data available on the Web via APIs, web services, and open data standards. For example, Pipl is a search engine designed to locate people’s information across the web. Pipl uses identity resolution algorithms to aggregate information and cross-link various sources before delivering an online profile containing a summary report of everything that’s publicly available for each individual. Pipl offers all that wealth of information to developers via an API.  Useful applications for this are, for example, running an API request for an email address; one can determine the user’s gender, age, location, interest, and provide an adaptive experience based on the individual user.

Pipl Search

Understanding the user story is possible with a network connection. However, network  connectivity is not only important to understand the user and his online record; it is a vital instrument that connects all other technologies together–cloud computing, understanding local weather, traffic conditions, or even the type of connection itself (Wi-Fi or G3) can help us understand context. Ultimately, the possibilities inherent in understanding and designing to the user’s story—his context—are possibilities built upon the collection of sensor data and user data via the network.

Sensor Data

A sensor for adaptive systems is any technology that allows a device to understand and evaluate context.  It includes the built-in accelerometer in smart devices, a camera, a clock, or even a microphone. We can use the various sensors embedded in smart devices to better understand the user’s environment. For example, the built-in accelerometer can be used to gauge if a user is walking or running.

There are two main scenarios for using sensors: everyday objects transmitting data like temperature or noise level to other devices, for example, iGrill, a cooking thermometer and app that communicates with smart devices via a secure, long-range Bluetooth connection. Or smart device applications, utilizing the built-in sensors to receive, process and output data to the user.  By using these sensors and mixing other technologies discussed above, we can often obtain powerful information on the context of use and utilize it to create adaptive systems.

iGrill Cooking Thermometer

Sensors can be a powerful design tool of the future. For example, with the aid of sensors e-commerce checkout will be as easy as logging into our bank account with no passwords. Here is an example of how using four layers of sensor data to secure with a degree of certainty the user’s identity so to create password-less banking that would present the user a “light” version of his bank account, so he quickly could check his account balance. Imagine a user is at home surfing to his bank account through his tablet computer. The first layer of security is the username associated with the tablet. Second is the location sensor, which will give us a greater degree of certainty that the user is in his home vicinity, cross-checked with his registered address with the bank. The third layer is the WiFi connection (its MAC address, a unique identifier assigned to a network) the user is surfing on. For the fourth layer, we can check for other nearby WiFi connection (The neighbors are sure to have unique WiFi MAC address) that can also be used as a security verification. If these bits of data are consistent across several password logins, the system can adapt and allow the user to enter without any password.

To learn more about adaptive design and how to get from sensors to context, it is highly recommended to read this paper by Albrecht Schmidt about Context-Aware Computing (Interaction Design Foundation Encyclopedia).

Conclusion

Today, we’re just starting to see the potential of using sensors and technology to connect between devices and people. The term “Internet of things” refers to uniquely identifiable objects that are network-connected, for example, a smart flowerpot that sends a signal when it’s time to water the flowers. There is no doubt that adaptive design will play a key role in making future devices and functional user interfaces that give users an intuitive control over their environments in any situation or context.

However, designing for device orientation brings various challenges and requires careful thinking. The experience must be as unobtrusive and transparent as possible, and we must understand the context of use for this functionality.

Nearly all mobile and tablet applications would benefit from being designed for device orientation. This article covers some of the basic concepts that designers and developers can use to add device orientation to their process. We’ll present some of the challenges when designing for device orientation, along with some solutions.

Useing Device Orientation for a Secondary Display

YouTube’s mobile application is a great example of device orientation design. Portrait mode offers a feature-rich interface for video discovery and the user’s account. Landscape mode provides an immersive experience with a full-screen video player and playback controls. When the video ends, the display switches back to portrait mode, prompting users to quickly tilt the device back and browse for additional videos.

YouTube mobile interface, portrait and landscape view.

However, using orientation to display a secondary interface can be confusing for users. For instance, in CardMunch (a business-card reader by LinkedIn), users can convert business-card images into address book contacts using the smartphone’s camera. Rotating CardMunch to landscape mode changes the interface altogether to a carousel overview of all of your saved cards.

This interface lacks any visual clues about its orientation, and it has limited controls. You are unable to edit or add business cards, making the carousel screen somewhat frustrating and confusing, especially if you’ve launched the app in landscape mode. In addition, the lack of visual clues in this landscape mode will deter most users from rotating the device and discovering the app’s other features.

CardMunch by LinkedIn. Missing visual clues for the orientation as a secondary display.

Categories Of Orientation Design

To help UX professionals and developers verbalize designing for device orientation I have categorized four main patterns for device orientation design:

Fluid:

The interface simply adjusts to the new orientation size.

In Skype’s mobile interface, the icons change position when the screen moves from portrait to landscape.

Pocket mobile interface. Same layout, different width.

Extended:

This interface adjusts to the screen’s size, adding or subtracting layout components according to the dimensions of the chosen orientation. For example, IMDb for the iPad uses the wider screen in landscape mode to add a filmography on the left. This list is also accessible in portrait mode by clicking the “All filmography” button in the middle-right of the screen.

IMDb Movies & TV for the iPad.

Adding visual elements and functionality to the layout in any orientation ultimately provides convenient choices for users. However, it is crucial not to force the user to hold a device a certain way, especially if the added functionality does not exist on the default device orientation.

Complementary:

With this interface, a changed orientation triggers an auxiliary screen with relevant supplementary information. An example would be a mobile financial application that displays data in the default portrait mode, and then provides a visual graph when the user rotates to landscape mode. The orientations show similar or complementary data and depend on each other.

An example of a complementary design interface.

Continuous:

Like YouTube, a continuous design enables the user to access a secondary interface by a simple rotation of the device. An example would be using a smartphone as a remote control for a smart TV. Rotating the device to landscape mode would reveal a full program guide, while also maintaining functionality for changing channels, browsing programs and recording future episodes.

A smart remote and program guide.

Considering the Default Orientation

Above & Beyond is an interactive eBook for the iPad about the life and work of the American caricature artist John Kascht. The beautiful art in this book is displayed in both portrait and landscape modes. However, horizontal mode shows important interactive elements that do not appear in portrait mode, such as a way to return to the main menu. Portrait is the iPad’s default orientation, so including this type of added interactivity only in landscape mode might confuse many users.

While portrait mode shows a detailed look at the art, and the interactive eBook provides some instructions at the beginning, a solution might be to allow users to tap the screen to reveal the menu.

A page from Above & Beyond. The interactive elements do not appear in the default portrait mode.

The default orientation for most smartphones and for the iPad is portrait. However, landscape is the default for Android, Windows 8 tablets and BlackBerry’s Playbook. To avoid confusion, remember that the primary orientation of your app should always serve the device’s default mode and functionality, not the other way around.

Understanding the Context

When designing applications for smart devices such as the mobile phone or a tablet, it is crucial to understand the context and circumstances in which that application will be used, particularly when designing for device orientation. Case in point, interactive cookbooks are now very popular, hardware and accessory manufacturers are creating devices to help us use the iPad in the kitchen, which include a washable stylus pan for use while cooking.

We can utilize orientation to create a better experience for the iPad cookbook. Using the default iPad orientation (portrait), users can flip through the book and view the full recipe and ingredients list for a particular dish. Rotating the device to landscape will change the interface to a cook-friendly layout with large buttons and step-by-step instructions. This cook friendly interface will also prevent the tablet from auto dimming and will allow users to flip pages by waving their hand in front of the built-in camera to avoid touching the screen with messy hands while cooking.

Understanding the kitchen context and how people cook with the iPad, makes the Interactive eBook much more functional in the kitchen and with the added consideration for device orientation, it creates a better overall user experience.

The Betty Crocker® Cookbook for iPad. An example of a cook-friendly interface.

Orientation Visual Clues

Creating an auxiliary screen or adding functionality, which depends on the way the user turns the device, could sometimes be counterintuitive to the user experience. Without any visual clues, a user could miss the added functionality on a particular orientation altogether. A classic example of a case such as this is the iPhone calculator, as demonstrated in this Salon post dedicated to explain how to change the iPhone calculator into a scientific calculator for the many users that could not find this added functionality.

When designing for device orientation, adding visual clues increases findability for the added display and creates an intuitive transparent experience for the user. Here are a few examples for device orientation visual clues:

Title bar

Affixing the title bar to the top of the default position in any orientation can create a subtle clue for the secondary display by urging users to tilt their heads (or the device) in order to read the text that appears on this bar. This technique is fundamental and should always apply when using orientation for a secondary display as a gentle reminder for the added functionality.

Note: This method will not give affordance to the added display on the default orientation (Portrait). In this case I recommend using Coach Marks, a quick tutorial or walkthrough video (when appropriate) to illustrate that the application is using orientation for a secondary display.

Toggle switch

Much like the universal icon for sharing or the Apple Action Button recognized for sharing, I propose using a standard icon as a visual representation for device orientation. You can download this icon here and use it freely.

The device orientation icon should appear at all times and used as a toggle switch between orientation displays. Using this toggle switch will not require the user to rotate the device for the added secondary view. However, it will eventually encourage using device orientation without pressing this button, because users will likely tilt the device in order to view the added screen. Turning the device back to the default orientation will automatically adjust the screen display. Ultimately, this icon will serve as a visual reminder for the added functionality, the user will not need to use this control to switch between orientation displays but simply rotate the device instead.

Here are examples using this toggle switch and icon:

Title bar toggle switch button.

If accepted, the device orientation icon could provide a visual clue giving affordance for using device orientation whenever it appears. Here is an additional example using this icon on a corner triangle toggle switch.

Corner triangle orientation display toggle switch.

Disclaimer: The device orientation icon is not a proven idea, and the value of adding a somewhat redundant UI control is debatable. However, in my opinion, it is a simple and effective idea, that will allow designers to rely on device orientation to enhance and extend any application. I hope that the solutions I propose here could spark a conversation within the design community so that, with a simple turn of a device, we can easily add essential functionalities to any application.

The drawer

The idea behind this concept is to create a drawer-like control that users can tap or swipe in order to see the secondary view. Rotating the device will automatically open the drawer, much like a curtain opening. By using animation to open and close the drawer, one can create a memorable effect for displaying data on device orientation.

Example of a drawer control.

Conclusion

Designing for device orientation is not new. On smartphones, orientation provides a larger virtual keyboard, and tablets email applications use a master–detail interface (Extended) when the device is rotated to landscape. However, device orientation design is widely considered as secondary functionality (to the main interface) because it is often missed by users using a device mainly on its default orientation. The main reason for this is simply because they are not aware of this added functionality if its available. By adding simple visual clues to the user interface, UX professionals can make the case for using device orientation to enhance the product and, ultimately, provide more engaging and user friendly experiences.

The Pipl search engine will aggregate the information and cross check it against various parameters, before delivering an online profile containing all of the available information collected on one individual. Unlike a typical search-engine, Pipl is designed to retrieve information from the deep web. The robots are programmed to interact with all searchable databases, extracting facts, contact details and other relevant information from personal profiles, member directories, scientific publications, court records and numerous other deep-web sources.

Pipl is not just about finding more results.  It uses advanced language-analysis and ranking algorithms to deliver quality results, focusing on the most relevant pieces of information. Results are presented in a single, easy-to-read results page.

I have summarized below parts of the process which were undertaken in designing this website.

The Home Page

The home page is unconventional because standard search engines usually only have one search field. Owing to the unique nature of this search engine, users are asked to enter a name and an optional location. One problem was that new users quickly scanning the site would be misled into thinking they were being asked for their own usernames and passwords, assuming they were expected to register for a service – which they were not.

The first attempt to resolve this issue, was to add a frame to visually combine the 2 fields. Next I removed the ‘Search’ copy from the button and replaced it with a search icon, so as to make it very clear to users, they were indeed about to perform a search.

Following several iterations, the framing was removed to create a more streamlined look and we add a series of smart visual clues to guide users through the process. The name field  is highlighted with a blue ‘person’ icon, and as soon as the user places their cursor over the area, the frame turns blue. When the user begins entering a name, the location icon  changes color to blue, together with the search button.  This indicated to the user that they can fill in the location information if they’d like to, or alternatively simply press the search button.

Search Results Page

Next, because Pipl is a search engine,  results returned links to external pages.  In the previous design many users were seeing the list of results and assuming that’s all they were. The challenge was to create an appearance that presented the results as links and encouraged users to click on them to access the (external) results pages.

Following a series of A/B testing, Pipl realized that using the same shade of blue used by Google on their links yielded a 20% increase in click through rate.

A decision was made to move to the ‘Google blue’. Examples of this can be seen below. A standardized search result would be in the new blue color and links were underlined. The full URL would then be displayed in green below. We removed the unnecessary icons and created a uniformed look across all results. Finally, in order to remove the clutter, we summarized the main categories above the results.

The result was that we created a more streamlined results page which felt less cluttered and was more intuitive for the user and felt like search results.

The previous color used for links had been drawn from the logo. A decision was taken to refresh the corporate logo for the new site and create a more contemporary look and feel. We have now used a color which will blend well across the overall site.

Searching the deep-web takes time, and search results could take several seconds. The loading bar in the previous design felt somewhat dated.

I designed a load bar which added some humor whilst the search was in progress. Users are more likely to be patient and respond favorably if they have a smile on their face while waiting. I also had the URLs from the source cycle through below throughout the loading process. This gave an indication that the process was working quickly and results were being found.

In order to provide high quality and relevant search results we need to understand the context of searches. Users most often were looking for their own online records or were searching for a lost relative. Pipl was able to offer a great service which enabled users to gain real satisfaction and accurate search results.

Below, you will find additional before and after screens from the project.

In traditional systems, the context of use did not change much. Whether the use was in the office or at a personal computer at home, the surroundings were similar and there was no need to adapt to different environments. In today’s world, smartphones, tablets, laptops, and smart TVs provide different services in different contexts. These services are consumed by a variety of users and require different interaction models, use cases, and planning. For this reason, UX professionals should first design for the context of use in order to provide better experiences and ultimately enhance the intended purpose of the product.

The Multiscreen App Ecosystem

Designing for context is especially important when designing for a multiscreen ecosystem, where multiple devices are all a part of one product.

Michal Levin, a UX designer at Google, describes multiscreen ecosystems in terms of three main categories. The first is consistent experience, where the application and the experience are similar across all screens. For example, the Google Search application provides the same search experience across all devices. The second category is the complementary experience, where devices work together and communicate with each other in order to create a unique experience. An example of this is Padracer, a racing game where the user’s iPhone serves as the steering wheel and the iPad screen as the racetrack.

– Padracer – a complementary multi-screen experience

Another example of a complementary multiscreen experience is SlingPlayer for mobile devices. A Slingbox is a device that allows you users to stream live television from their living room to mobile devices. Using the SlingPlayer application, users can watch or record their favorite programs from a remote location. Each device complements the other and the experience could not exist with just a single device.

– A screenshot from SlingPlayer for tablet

The Continuous Multiscreen Experience

The third category of app ecosystems the continuous multiscreen experience, which is possibly the most important category for a contextual multiscreen design. For a continuous experience across several devices, UX professionals must evaluate when and where a product will be used in order to assess the optimal experience for the user at the time of use. AllRecipes is an example of a continuous ecosystem where one can search for recipes online using a desktop computer and add ingredients to create a shopping list. While shopping, this list is accessible using a smartphone, the device most likely to be carried while shopping. The app also allows one to scan products using the camera and receive additional recipe ideas with greater ease. When it’s time to cook, an iPad can be brought into the kitchen for a cook-friendly interface with large fonts and big buttons so the user can touch the tablet with a knuckle in case of wet or messy fingers. The tablet’s larger screen is a perfect size for the kitchen, much like a traditional cookbook resting on the counter.

– Cook friendly interface with *large buttons for messy hands.

*Understanding the context is vital, although the large buttons work well, simple voice commands for “next” and “back” could be a better solution.

Eventbrite is another great example of a continuous multiscreen ecosystem. Users can register for an event through the Eventbrite website and receive an email ticket that they can print. Alternatively, users can download the mobile app for easy access to tickets and event information. The mobile app is ideal in this case because users will most likely take their smartphones to events to use a unique QR code as a ticket, which the app displays in high brightness and contrast for easy scanning in dark environments.

But the Eventbrite experience should not end there. The makers of Bizzabo, a new app designed to improve the social experience of conferences, believe in the importance of eye contact, handshakes, and face-to-face meetings. Their mobile app allows event organizers to import Eventbrite event information and engage directly with their attendees, while enabling professionals to network efficiently at conferences and meetups. This continuous experience is a natural evolution to the Eventbrite platform.

– Eventbrite and Bizzabo, a continuous experience.

Additional Categories of Multiscreen Ecosystems

Precious, a studio based in Hamburg, Germany published a collection of additional multiscreen categories and scenarios to help understand and define strategies for the multiscreen world. It’s a handy reference for discussing solutions for digital products and services.

Understanding The Story

Sarah Doody writes, “A product is more than an idea, it’s more than a website, and it’s more than a transaction or list of functionalities. A product should provide an experience or service that adds value to someone’s life through fulfilling a need or satisfying a desire.” Understanding how people use a product and telling their stories helps designs create better critical paths and experiences for each platform when designing for a continuous multiscreen ecosystem. For example, a social network might decide that image sharing plays a significant role in developing the value of the product by enhancing user engagement. Although the website is the perfect platform for sharing and keeping in touch with friends, the critical path in the smartphone application can be directed towards the camera. Smartphones are carried everywhere, so a camera is always at arm’s reach. The images can then be synched with a tablet, which is an ideal platform for image sharing and manipulation because of its larger screen. Those images can be streamed directly from the tablet to a smart TV when friends come to visit, rounding out a great value for people who use the entire ecosystem.

Contextual Paths

When we understand how context affects usage for each device, we can create contextual paths to help guide the design for a multiscreen ecosystem. For example, a responsive website for a large city tells visitors what to do when in town, shows maps of public transportation, and provides directories of municipal services. To adapt to the context of a mobile platform, this same website should provide different paths according to the context to accommodate users on the go, typically by focusing on location-specific information such as bus routes. To determine these paths, designers can utilize web analytics to evaluate what users look for when using different platforms.

Device for Context

After doing some research I have accumulated experience in understanding the basic context assumption for the various device types:

The Smarthphone

The idea that smartphone users are generally on-the-go outside the home, using the device as a substitute or a replacement for the personal computer is a misconception. Rather, smartphones are becoming a platform of their own and not a substitute for the computer, both at home and on-the-go.

However, it is safe to assume that the smartphone is always within reach throughout the day. With the rise of other convenient devices for media consumption (e.g., tablets), users mainly use smartphones for micro-tasks: looking for quick information, killing time, and social sharing.

Because the smartphone is always within reach, the context of use could be understood in terms of the application’s intended purpose. An example of this is Evernote, a multiscreen ecosystem that allows you to take notes, create to-do lists, clip articles, and save URLs. Evernote makes these notes completely searchable whether you are at home, at work, or on the go. In its mobile app, Evernote understands the mobile context and provides quick access for text notes, photo capture, and voice dictation. By utilizing the device capabilities and providing a quick way to save ideas on-the-go, Eventbrite turns smartphones into a great note-taking platform.

Our Mobile Planet is a great resource for understanding the mobile consumer. The insights within this website are a result of an extensive technology research project commissioned by Google for understanding the smartphone adoption and usage around the world.

The Tablet

Tablets are mostly used for media consumption, reading books, browsing news, and checking emails. People may use smartphones to look for a quick article to kill the three minutes waiting for a train; once on the train, they may take out the iPad for the hour it takes them to ride home (Usability of iPad Apps and Websites). Tablets are used in environments not conducive to a keyboard and mouse such as lying in bed, watching television, or standing. Because of its lighter weight in comparison to a laptop, the tablet is easily carried; however, its main usage is within the home. Most tablet owners use the device daily for an average of 90 minutes a day, mostly on weekdays.

Because the screen of a tablet is larger than that of a smartphone, the tablet is ideal for reading and viewing media such as images and movies, and the larger virtual keyboard is more convenient for writing emails and note-taking. However, the tablet is not currently considered a work tool for most people, nor is a replacement to the desktop computer; it is merely a more convenient way to consume media and surf the Web away from the office desk or the typical computer environment.

As technology advances, future tablets will have the power to replace the laptop PC. With the upcoming release of Windows 8, an operating system geared to touch devices and personal computers, there will most likely be a shift in the market, and perhaps one day the tablet (with an auxiliary keyboard) may replace the laptop altogether.

Tablets are a perfect platform for reading. Pocket (formally known as Read it Later) is a great example of an app that uses the tablet to enhance a consistent multiscreen experience. When the user finds something on the Web while using a desktop PC or laptop that he wants to view or read later, he simply puts it in Pocket. Pocket captures those reading items and displays them in a clean, easy-to-read layout on a tablet (or a smartphone), creating a personalized magazine for the individual user.

The Personal Computer (Desktop or Laptop)

People use personal computers for two completely different functions: for work and for leisure. Because personal computers are found in almost every home and office, it is essential to consider them when designing for a multi-screen ecosystem. Oftentimes, users will look for an application or find something they’re interested in while surfing the Web on a personal computer, and then continue the interaction onto other devices. Personal computers are also used to backup files and images, and users often revert back to the computer for the most important tasks in their daily routines.

Mouse and keyboard ergonomics and the advanced processing powers of personal computers provide an efficient working environment. When designing multiscreen ecosystems, UX professionals can utilize additional devices to serve as auxiliary screens. For example, with Adobe Nav and a network connection between an iPad and a computer, users can move the Photoshop toolbar onto the tablet and customize it to more easily access the toolbar without crowding the main interface, creating an even more efficient environment.

The Smart TV

Smart TV is a general term for a new generation television viewing experiences where the TV (or a specialized device connected to it, such as an Apple TV) has Internet connectivity and can run applications (e.g., Netflix) and video processing utilities.

Designing for television screens is also known as “designing for the 10-foot experience” because, from the distance of the living room couch, the apparent size of elements on the screen is noticeably smaller compared to a computer screen, where the user’s eyes are typically less than two feet (60 cm) from the display. When designing for the context of smart TVs, one can assume that users are in a very comfortable setting in their living rooms, often wanting to relax, to be entertained, and to interact with people who are in the room with them. The interaction with this device could last a few hours or longer.

The preferred television input device is a remote control. For the most part, people don’t want to use a mouse and keyboard when they’re watching television. This means the UI has to be simple and obvious, and users must be able to navigate with their thumbs without looking at the input device.

With devices like Google TV or Apple TV using AirPlay, smart TVs can integrate with other devices such as smartphones and tablets. For example, a tablet can be used as a program guide and as a way to read more about the program currently being watched. Or for a multiscreen experience, a user can begin watching a show on her laptop and then switch to her television for a more comfortable environment. The move between each device is synced and seamless to the user.

Following up his 2011 article Re-Thinking User Interface Design for the TV Platform, Jim Mischel noted in an interview: “A good way to satisfy the television user is by presenting a simple user interface on a tablet or phone. The tablet/phone presents the interface, and the user’s input on that device controls the television. Note that this is only important when dealing with short-format content. If the user is going to watch an hour-long television show or a two-hour movie, then Web interfaces like Hulu, or cumbersome cable TV-like interfaces are okay. The user is willing to put up with a little aggravation if the reward is an hour or two of uninterrupted entertainment. But if the user is watching YouTube videos or consuming other short format content, the user interface has to be simple. Really simple. The user can’t be bothered with having to go through a ton of menus every three minutes.”

The television could also serve as an extension to other devices, using the large display for streaming media or images from the tablet or using a high-quality sound system for better audio experiences. A good example of this is Sting 25, an app celebrating the life and work of the musician Sting. It allows a dual-screen AirPlay viewing of exclusive concert footage wirelessly on a smart TV while exploring additional content on an iPad.

A natural user interface (NUI) is an emerging concept in human–computer interaction, and refers to an interface that allows a user to control a tool or digital content using relatively natural means of communication. The goal of NUI is to make computing and technology simple to use, everywhere around us, accessible to people with minimal technical expertise, reliable and more intuitive. With a motion-sensing input device technology like Microsoft’s Kinect, the living room environment becomes a setting for an even greater TV experience. For example, face recognition software could detect a person and flip through his favorite channels as he taps two fingers together or waives his hand, and a voice command could initiate more complex actions.

Here is a demonstration of gesture controls by PrimeSense, an Israeli Company specializing in NUIs:

Conclusion

Designing for context is about aligning the purpose of a product with the user’s requirements at any given time or in any given situation. By evaluating each platform against the story of the users and how (or where) they use the product, designers can create better flows across and within the ecosystem. Creating these product ecosystems turns smart devices into smart displays. These smart displays follow users throughout their day and help drive the experience, allowing users to engage with the product more than ever before.

The experience of interactive eBooks should not be confined to animations based on touch-and-response interaction, or merely flipping the page; when designing these Books one must ask what is the enhanced experience—why to move from print to digital, and how to create value and fun.

Interactivity for the Sake of Interactivity

If a book app does not use interactivity in order to enhance the reading experience, it does not belong in the interactive eBook category. In the race to bring interactive books to market, some of the books have only featured very superficial interactivity—what I call “interactivity for the sake of interactivity”—where, for example, touching an image activates a simple animation such as making a butterfly fly, or a tree drop leaves to the ground. These interactive experiences do not add value to the story, and are therefore somewhat meaningless.
 
There are a few exceptions where this type of interactivity is actually a success. For example. one of the first books published as an interactive app for the iPad was Alice in Wonderland. This book was a phenomenal success though offered nothing but eye-candy interactivity. When the app was first published, the reviews called it “a reinvention of reading” that made clever use of the accelerometer to make Alice grow as big as a house or to throw tarts at the Queen of Hearts and watch them bounce. Although these activities through the 52 pages of the book are fun, I think they distract from the actual story. The reason this book was such a success is due its having been published when the iPad was fairly new, and touch interactivity was still an exciting experience.
 

 
 
Another book that was fairly successful at the time was The Pedlar Lady of Gushing Cross, which offers narrated animation with very basic interactivity, but was considered revolutionary when it came out because reading the story while seeing the animation unfold was definitely an enhanced experience to the young reader. However, this book did not offer any real value through interactivity, and might as well be classified as a short animated movie. The limited interactivity of seeing letters animate while you tilt the device was merely a gimmick, as you can see in the video below
 

 
 
Cozmo’s Day Off is an interactive eBook that was on the top-seller list for many months, and is packed with interactive elements that made it a great success. It contains over 100 unique audio and animated interactions. However, this app would be better characterized as a game for young kids and not as an interactive storybook because the story seems secondary to all the bells and whistles, and it’s written in style not intended for young audiences. But perhaps this is a case where interaction simply for the sake of interaction can be the whole point of a book.
 
The image below shows all of the hotspots that trigger an animation sequence for one page of the book:

 

Interaction for Value

It is possible for interactivity to go beyond the superficial, to add value to the book and create an experience that would be impossible in print. Here are a few examples of such cases.
 
Al Gore’s Our Choice is a great example of how meaningful interactivity creates an engaging and fun learning experience. With clever use of interactive infographics, animations, documentary videos, and images, this book is a great example of what the future has in store for digital publishing.
 

 
 
The Martha Stewart Cookies iPad app is a wonderful example of an interactive recipe book. Besides just offering great recipes, it also allows you to search recipes based on ingredients and cookie type to find the perfect cookie for your needs. For example, you might use the app’s search wheel (below) to look for bars and biscotti-type cookies with oatmeal as the main flavor component. This is a great added value because this type of interaction is unmatched in print.
 

 
 
Paris: DK Eyewitness is probably the most complete travel guide you can find for the iPad. It features beautiful cutaways of buildings that can be explored by tapping and zooming, complete offline maps for all the central districts of the city, interactive city and park walks with “hotspots,” and extensive listings of the best sights relative to your current location. No more searching aimlessly for your location on a map or looking through index pages; the interactive app shows what’s around you within walking distance, making the iPad a must-carry on in your travel bag for an experience unparalleled in a traditional travel guides.
 

 
 
Bobo Explores Light is an educational experience for young adults. It puts a fully functional science museum in the palm of your hand, teaching you about lasers, telescopes, lightning, reflection, bioluminescence, and sunlight. This is great example of using simple interactivity to explain relatively complex topics through science experiments that you can actually perform on your iPad. Bobo, a friendly robot, serves as a guide, taking the young reader through space, land, and sea, to learn all about the science of light.
 

 
 
In my book, Timor the Alligator, kids participate in the story by picking toothpaste and helping Timor brush his teeth. This story could not have been told in a printed book because, without the use of interactivity, young kids would not be able to visually understand that brushing actually helps keep a clean mouth. The simple process of choosing a toothbrush, adding toothpaste, and brushing Timor’s teeth until they turn white serves as an educational experience for preschoolers and toddlers reading the book.
 

 
 
With the Numberlys app, kids (and adults) learn about the alphabet through a series of fun interactive games. This book probably has the most spectacular visuals I’ve seen to date. Its aesthetic is inspired by Fritz Lang’s silent film, Metropolis, so the app offers a unique cinematic experience and gameplay to engage users to learn about the (fictitious) “origin of the alphabet.”
 

 
 
As you can see from these examples, interactive eBooks are no longer just about a touch-to-animate type of interactivity, nor simply the touch interface controls. Rather, they are about adding value through interactivity by using the full capabilities of a touch device to engage the user and enhance the learning and reading experience. These engaging experiences are what I call a true reinvention of reading.

I recently attended a conference about the upcoming Windows 8 user interface from a UX perspective. The meeting held at the Microsoft’s headquarters is part of the company’s effort to accelerate application design for this new platform.
 
The talk by Lior Yair, a senior UX Designer inspired me to think that the new Windows 8 Metro (design language) is really about content first, traditional navigation and the application shell is now visually obsolete.
 
The concept and the philosophy for “content first” is not new when it comes to interface design but Microsoft really braved this idea forward and did a great job doing so. Content is now king when designing applications for the Metro UI; allowing people to interact directly with what they want to see helps design better experiences.
 
Here is an example:

RSS Reader app. Before and After. The area in green marks the content area. In Windows 8 users will focus on content instead of trying to work around the navigation.
 
In the example above you can see the RSS Reader is stripped away from any navigational components and presented in a clean well structured environment relying on the actual content to also function as the main UI.
 
Additionally, the Windows Metro UI is specifically designed for Tablet and Personal Computers and offers advance thinking when it comes to these platforms. After seeing a demonstration of Windows 8, the iPad iOS feels now more than ever like an upgraded version of the iPhone iOS and not designed specifically for the larger screen. I say that with a great conviction because often I encounter UX blunders when using my iPad, for example the back button at the upper left corner is inconvenient to reach or even the simplest jump between two running applications takes extra clicks that the metro UI does not require. Furthermore, the new Windows UI offers a split screen view so you can run two applications at the same time with what they call “Snap View”.
 
Microsoft has created extensive documentation for developers and designers. I recommend reading more about this topic here: http://msdn.microsoft.com/en-us/library/windows/apps/

Designing Skype for Windows 8

I was motivated to investigate this topic even further, as a personal challenge I decided to brave and design Skype for Windows 8. I chose Skype because it’s a familiar program and hopefully people could relate to my design through their own experiences.
 

Login screen. An opportunity to let the Skype brand shine.
 
 

Skype for iPad. The green marker shows the content area within the iPad user interface.
 
 

My Design for the Home Screen. Focusing on recent conversations and easy access to the Contact List.
 
I felt that the content and substance of Skype is the Contact List. The people that you communicate with is the most important feature, and that’s why I created a home screen that allows you an easy overview of all your contacts and recent conversations.
 
 

Contact Page. An overview of recent communications.
 
 

Call Screen. Although the icons at the top right would be a perfect fit for the bottom App Bar I wanted to have a visual overview of the critical interactions on this page.
 
 

Windows 8 Snap View allows you to surf the web while having a conversation.
 
 
In conclusion
If you plan to design applications for windows 8 I recommend considering the following points:

• Content is king
• Think minimalist. Less is more.
• Metro is only framework and guidelines, it’s ok to think outside the box.
• Negative space is positive. Spacing is your friend. Keep a consistent space between elements and always align.
• Use typography to create structure and hierarchy
• Present only relevant content for each screen.

 
Persisting with the Metro design language Microsoft has quite possibly created a revolution in interaction design and I definitely commend them for that.
 
You can read more about the Metro design principles here:
http://www.microsoft.com/design/toolbox/tutorials/windows-phone-7/metro/
 
UX at Microsoft from a designer point of view:
http://ux.artu.tv/?page_id=190
 
 
 

Kids are curious and eager to explore. They tend to touch everything they see on the screen and notice things adults sometimes miss. For example, they don’t experience banner blindness; for them, a banner is an exciting opportunity to learn what will happen if they touch the screen. Most interaction problems for young users occur when they are forced to navigate away from what they want to do because they unintentionally trigger an action they did not intend to. Banner ads, a swipe gesture, or even a simple menu button are sometimes causes for aggravation for the young user and the observing parent.

Kids explore an interface by touching almost aimlessly on the screen, trying to discover as much as they can. A banner ad or a Facebook “like” button may work for some applications, but when designing for kids we must realize that banners link to pages toddlers cannot return from, and pressing “like” takes them out of the app and into to a browser with the Facebook page.

A Childproof Navigational Interface

A similar usability problem happens when a young user inadvertently touches a menu button. For UX designers and developers, making it easy to access menu controls at any point in the app is a normal part of making easy-to-navigate interfaces; for inquisitive kids, however, a menu button visible on all screens can sometimes be a source for frustration.

When designing for toddlers and preschoolers I recommend creating a “childproof” navigation interface to prevent children accidentally brushing or touching the menu while using the application. The idea is simple: design menu access for the parent only. Just like a childproof safety cap for a medicine bottle or other devices made safe for young children, we create a menu system that is childproof.

For example, a childproof app might require that a menu button be tapped once to change its color, and then tapped again after a short interval to activate the menu. This short delay gives the child an opportunity to divert attention to something else on the screen. If the child presses once without any rewarding feedback, he is not likely to press again. Another example is a banner type graphic that you must “pull” (swipe down) in order to see a menu, or even basic linear navigation flow without a menu to allow a focus on the tasks at hand with no unwanted interruptions.

Here are a few interesting examples of apps designed with kids in mind:

Heydooda! The kitty says: Hello animal kids

–Tapping the home button in the lower left side will slide the button to the upper left side.

Bartleby’s Book of Buttons Vol. 2

–Dragging the red banner down activates the menu.

Roxie’s Doors

–Dragging the red arrow down displays menu options.

Nighty Night! HD

–Basic linear navigation with no menu access

Gestures for Young Kids

Sometimes a swipe gesture is used to flip the page in interactive storybooks, but for young children trying to interact with the story, flipping the page may not be what they’re trying to do. For an inquisitive child, a swipe is sometimes a clumsy press or an attempt to move things around the screen; “next” and “back” buttons to change a page would be more effective in this case especially when combining other interactive elements on the page. Generally, I recommend avoiding any sensitive triggers when designing for young kids.

A Present for Milo app is a good example of a confusing experience for children using the swipe for navigation. Although it’s a fun book to read, tapping the interactive elements on the screen sometimes flips the page unintentionally and distract from the story.

–Tapping the interactive elements on the screen unintentionally flips the page

Consider the Device

Accelerometers and young kids don’t go together. The tablet is an expensive device, so invoking accelerometer triggers as means to create interaction for kids should be avoided, as children don’t have a strong grip and tend to drop things. Shaking the device to erase a drawing or tilting a device up to play a game could be a cause for disaster.

Toy Story Read-Along

–Using the accelerometer could be dangerous.

Kids love playing together, so applications for kids should allow multi-touch for a more enjoyable experience. For example, a drawing app should allow painting with multiple fingers to allow two kids to draw at the same time. Allowing multi-touch will also make the interface more forgiving when a child is holding the device. If the child is touching the corner of the screen with one finger and tapping with another, he would be frustrated when nothing happens because the application only allows for one-finger touch. Luke Wroblewski writes about this in his article Touch-based App Design for Toddlers.

Toca Boca Birthday Party Playtime

–Toca Boca Birthday Party Playtime allows two participants to play at the same time.

Luke also writes about how interface elements sometimes intrude on the experience for young kids, replacing large, colorful buttons and appealing intuitive controls with a generic design that children cannot relate to. Here is an example:

Toy Story Read-Along

Toy Story Read-Along presents kids with options they don’t understand.

Hush Little Baby

–Hush Little Baby – a setup menu kids cannot use

The iPad’s five-finger pinch-to-terminate gesture, or the four-finger swipe are definitely causes of frustration for young users. Additionally, the iPad Home button and the Android Menu Bar are not childproof, as they are always visible and invite a curious touch. Apps for kids should allow a quick return to the application in case these buttons are pressed. For easy recovery, avoid displaying a logo splash screen and menu each time the application starts.

 
Below you will find some of our work for logo and brand development.
 

 

Our Design

Original Sub Page

Our Design