he promise of write-once-run-anywhere is arguably more attractive and far more economically advantageous for developers targeting mobile devices; mobile hardware is so diverse that cutting development time by writing one version of the application?mobile developers’ holy grail?is highly desirable. In practice, however that quest is rarely achievable, but you can minimize the time and effort necessary to target multiple devices if you plan ahead.
No application design decision is more apparent to your customers or target audience than the user interface?and the user interface is potentially the most difficult to address in a wireless application. Although the MIDP profile provides a standardized environment on which to write applications and a set of classes to help you create user interfaces, the actual availability, appearance and functionality of interface widgets varies substantially from device to device, even between different device models. Further, environment constraints, such as memory, screen real estate, and persistent storage availability, inevitably prevent you from displaying large amounts of data on a screen.
This article presents some of the concerns, pitfalls, and obstacles in creating a user interface for an application expected to run on diverse devices
|Author’s Note: Although I prefer working with IBM’s WebSphere Studio Device Developer. I like to use Sun’s Wireless Toolkit version 1.4 to further test my applications. I used the wireless toolkit and a simple text editor to create the project for this article, making extensive use of the toolkit’s memory and network monitoring facilities. The toolkit also comes with a very complete UI Demo application that you can use to see all of the UI widgets discussed in the first part of the article. You can also use this application to see how different user interfaces look on different devices.|
Because this article focuses on UI problems, I didn’t worry too much about network data access or data parsing, but in real-world applications intended for a variety of devices you should try to minimize the total size of the application and memory use.
Forget AWT and Swing in MIDP
Desktop Java programmers are familiar with AWT or Swing for building UIs, but you won’t find either of those in a MIDP programming environment because of the size and memory constraints on the majority of handheld devices. At a more fundamental level, concepts such as windows, point-and-click, drag-and-drop and other desktop UI metaphors don’t exist in MIDP either.
So, when programming user interfaces for CLDC profiles, you’ll have to forget your reliance on layout managers, primitive component spacing, custom interface development, and assumptions about the look and feel, functionality, and execution of your interface elements. If you’ve worked with AWT you may be able to transfer some of that experience over to handheld device UI development.
J2ME’s User Interface Facilities
For device portability, one segment of the J2ME API, called the “High-Level User Interface API,” uses a high abstraction level to provide developers with a set of UI components. This high abstraction level lets applications maintain consistency in style and functionality with UI components provided natively by the device, but it’s also a tradeoff because it leaves the J2ME application developer with little control over many aspects of the user interface. You’ll need to plan accordingly to overcome the many obstacles that such limited functionality places on your development.
Fortunately, the J2ME specification also provides more direct control of the screen through the Low-Level UI API. This set of classes provides almost direct access to the screen space allocated to the application. For example, you can use the classes in this segment in applications such as games to draw primitives, including circles, squares, lines, images, to clip portions of the screen, etc.
You can’t use both the high- and low-level interfaces together though, so if you need to use a standard UI control provided by the High-Level UI API while using the Low-Level UI API your only option is to reimplement such a control. Reimplementation might be appropriate for a specific application but might also be non-portable.
For the sample application we’ll stick with the High-Level UI API for maximum portability.
Take a look at the UIDemo application provided by Sun’s Wireless Toolkit while you read the next part of this article.
What’s in the High-Level UI API?
The concepts and decisions behind the high-level API are consistent, but their function, display, and placement differs on different devices. The minimum MIDP device requirements are:
- Screen-size: 96×54
- Display Depth: 1-bit
- Approximately Square Pixels
- Input (one or more of the following user-input mechanisms):
- One-handed keyboard
- Two-Handed Keyboard
- Touch Screens
- 128 kilobytes for MIDP components
- 8 kilobytes of non-volatile memory for persistent storage
- 32 kilobytes of memory for the Java heap
- Two, wireless, intermittent with limited bandwidth
MIDP approaches user interfaces using a metaphor of Screens as opposed to the common Window concept used for modern desktop development. MIDP assumes that the target device can display only one screen at a time. It doesn’t even provide controls automatically for users to navigate between screens?you have to provide users with buttons or other methods for navigating through multi-screen applications.
The abstract Screen class lets you specify a title and text for a ticker, an item that scrolls text horizontally across the screen (exact ticker placement is specific to the implementation).
|Figure 1: The image on the left shows a Form as rendered on a cell phone. On the right is the Palm OS implementation. Notice the difference between ticker and label placement.|
With the high-level UI API you don’t handle user events directly; the API captures these commands and either handles them internally or wraps them up as objects that it passes to you for handling. That’s good, because you don’t have to worry about device-specific navigation, which keys to trap, or how to pass focus to the correct interface item. Instead, you can concentrate on the higher level API interface flow. You’ll also find that you can’t draw directly to a Screen or its derivatives, but I’ll show you a way to display dynamic graphics later in this article.
Screens run under the control of the MIDlet’s Display class. To get a reference to the Display class, call the Midlet.getDisplay() method. The object returned from getDisplay is the same throughout the lifecycle of the MIDlet so you can store this value for reference later in your application.
After obtaining the Display instance, you set or remove the currently displayed Screen by calling the Display.setCurrent(Displayable) method. Calling getCurrent returns the currently displayed Screen.
Calling setCurrent replaces the current screen. Calling setCurrent with null removes the current screen. Calling setCurrent, however, doesn’t guarantee that the specified Screen will be available when the method call returns; the decision as to when to replace the current screen is left up to the device and the implementation. In other words, performing a potentially time-consuming processing after calling setCurrent might make your application appear to be frozen. To solve the problem, perform such time-consuming tasks in a separate thread. For example, if your application performs a lengthy network lookup, do the lookup in a separate thread to avoid locking up the device display.
MIDP uses the abstract Screen class as the base class from which other screens in the high-level UI API are derived. Screen, in turn, is derived from Displayable. Other displayable classes, such as forms and user interface widgets (called Items), are described below.
The concrete class Form adds to the Screen class the ability to append Items, and provides the ability to create Form-like screens for devices. The MIDP specification describes how a device should lay out items on a form. For more recommendations, you should check out the MIDP 1.0a Style Guide. Links to both documents are available at the end of this article. Although the suggestions are general they can help developers and UI designers construct usable forms. The specifications state that:
- Items appear on the Form from top to bottom in the same order in which the internal list maintains the items.
- Forms can scroll vertically but not horizontally. Items that exceed the form’s width will be wrapped if possible, as in a string item, or clipped, as in an image item. Scrolling is device dependent and could be implemented as a scroll bar, as up and down arrows, as in the case of a Palm device, or as a small arrow near the bottom of the screen to indicate that more items exist below the visible area.
- All items appear on a line of their own except for StringItem and ImageItem.
- All items can have an associated label and if one is provided the label is displayed somewhere near the item and may be rendered in a way that distinguishes it from the item.
- Multiple StringItems or ImageItems in succession that do not have Labels associated with them are laid out horizontally whenever possible. If there is insufficient room the items are broken and word wrapped or clipped.
- If a StringItem contains a line break, the line break is displayed.
The high-level UI API provides a variety of Items that you can add to forms. You’ll have to add code to provide your user with interactivity but you’ll find that for most basic UI data entry and display requirements these items will do the job.
The high-level UI API provides a variety of Items that you can add to forms. You’ll have to add code to provide your user with interactivity but you’ll find that for most basic UI data entry and display requirements these items will do the job.
All Items are descendents of the abstract Item class, which means that they all have an associated label. The High-level UI API provides the following items:
- StringItem. A string item displays a non-editable string and its associated label. You can change the text programmatically with the StringItem’s setText method and retrieve its value via the getText method.
Figure 2: Notice the difference in how the label is rendered on a cell phone in contrast to how the Palm OS renders it.
- ImageItem. Displays an image and its associated label, or an alternate string if the image cannot be displayed. The image must be immutable and can be set using setImage. You can provide layout hints to the application to specify how the image item should be laid out (see the documentation for the layout hints).
- TextField. This displays text in an editable field. You can specify a pre-existing value through the constructor or with the setString method. The text field can be a single or multi-line text field, although how each device handles this varies. Some allow multi-line editing directly on the containing form, while others require a separate screen. You can specify how the field will handle data entered by the user, or constraints, but devices are not required to provide all of these. These constraints provide the user some guidance for what data to enter and potentially ease data entry. The following constraints are available:
- ANY – Allow the field to contain any character
- EMAILADDR – Contrains the input to any characters valid in an email address
- NUMERIC – Restricts the input to numeric characters only
- PASSWORD – Doesn’t restrict character entry but tells the device to use any password protection facilities available for user input such as not echoing back characters or replacing characters with asterisks.
- PHONENUMBER – Constrains the field to valid phone number formats or characters
- URL – Restricts the field values to those available for Uniform Resource Locators such as http://, ftp:// etc.
Figure 3: The cell phone renders both interactive and non-interactive gauges without specifying the current value, while the Palm OS renders the current value for interactive gauges.
A TextField notifies ItemListeners (see the next section for details) later) of content changes provided by user input but not about content changes made programmatically.
- TextBox. Provides text input somewhat like a TextField but a TextBox takes up the full screen and does not notify ItemListeners of content changes.
- DateField. Allows user input of a date and/or time value. Internally, MIDP holds the value as a java.util.Date object. You can specify an initial date or accept the default null date value. How the device handles and renders user input is device dependent and may be anything from a simple text entry to a complicated calendar or clock widget. The DateField notifies ItemListeners of any changes.
- Gauge. You use a gauge to display a continuous value over a specified range. The gauge can be interactive, letting users change the value manually, or non-interactive, appropriate for tasks such as a progress bar.
- ChoiceGroup. Provides a list of alternative choices to the user and allows one or more choices to be selected. ChoiceGroups can be EXCLUSIVE, like a radio group in standard desktop applications, or MULTIPLE, allowing multiple item selection. ChoiceGroups notify ItemListeners of changes when the state of a choice (selected or not selected) changes, but not when focus changes from one choice to another.
- List. A list behaves in the same manner as a ChoiceGroup but provides the list in a separate screen and adds the IMPLICIT selection mode in addition to MULTIPLE and EXCLUSIVE. The implicit mode sends a List.SELECT_COMMAND to an associated CommandListener when a user selects an item from the list.
The preceding interface items are guaranteed to be available on all devices supporting MIDP. With only these basic building blocks to work with you have to keep in mind that data entry and navigation can be difficult on some devices. Plan your application to help the user navigate successfully and easily through your application.
Listening to Item Changes
Your application UI may require dynamic updating based on values provided by the user, or may need extended data validation beyond what’s automatically provided by some devices. The High-Level UI API lets you specify ItemListeners?objects notified of data changes based on user input.
To listen to item state changes you implement the ItemStateChange method of the javax.microedition.lcdui.ItemListener interface and register it with a Form instance by passing your class to the Form’s setItemStateListener method. Subsequent calls to setItemStateListener replace the existing listener with the new listener; only one listener can be registered at any one time. The item that generated the event will be passed in as an argument, so you can use the item’s accessor methods to get and change the data.
You need to be aware that not all items notify item listeners. In addition, the conditions under which items notify listeners are item-specific. The following items do notify listeners:
- TextField. The specification requires only that a TextField notify listeners of changes no later than after the focus moves away from the item. Unfortunately, the actual time at which the ItemListener is notified is implementation-dependent.
- Gauges. Gauges notify ItemListeners of state changes every time the value of the gauge changes. Some implementations of the Gauge widget don’t render the current value, and a change in the value might not be noticeable on the gauge itself, so tying notifications to updates of a StringItem can provide the user with the exact value of the Gauge. On the other hand, if the value is rendered in that platform then by using a separate StringItem to track changes, you’re duplicating information on the screen and potentially taking up valuable real estate. Regardless, to be on the safe side I recommend always adding a peer StringItem to clearly specify the value.
- ChoiceGroup. ChoiceGroups come in two flavors?multiple and exclusive&3151;consequently, they notify ItemListeners differently based on the type of choice group. In multiple, group notification occurs when an element is selected or deselected. In exclusive, group notification occurs when one element is selected. Even though any currently selected choice in an exclusive group will be deselected when a user selects a different choice, the item issues only one notification per change.
Now that you know how to add items to a form and listen to changes on user input you have to provide a way for users to navigate between the various screens in your application. You achieve this with the use of Commands associated with forms. You implement the CommandListener interface to respond to commands issued by the current form.
|Figure 6: On the left, the two available commands, Back and Quit, are mapped to the softkeys typically available on cell phones while on the Palm OS the commands are rendered as buttons on the screen and also added to drop down menus.|
You construct commands from the concrete javax.microedition.lcdui.Command class, passing its constructor a label, a type, and a priority. The command uses the label for display. The type and priority provide placement hints to the device. If you associate more than one command of the same type with a form, the device uses the command’s priority to determine placement. You add the command to the Form (or Displayable) by calling the Form’s addCommand method. You can remove Commands by calling the removeCommand method. You can share the same commands between forms.
How devices render and place commands varies widely from device to device; some cell phones will map two commands to the softkeys on a cell phone and put the rest in a menu that users can access from one of the softkeys. In contrast, a Palm Pilot will place the commands into drop-down menus or on-screen buttons. You have no control over command placement, you can only provide hints to the device.
Listening to commands on a screen is simple. Implement the CommandListener interface and put your command processing code or action delegation in the commandAction method of the class. The commandAction method receives the Command object to which it should respond and the Displayable associated with the command. You set the current command listener for a form by calling its setCommandListenermethod. You can have only one CommandListener associated with each form. Setting a new CommandListener replaces the existing one; passing null to getCommandListener removes the current Command.
Approaching UI Development
As developers, we’re used to?and often demand?that functionality be available at the click of a button, as a quick key combination, or easily accessible from a list of commands in a menu. In the constrained environment of a mobile device this is often not useful and can, in fact, be overwhelming to a user. Personally, I’m often perplexed at the ergonomic and human interface choices made for interacting with devices, so I take the approach that the fewer commands available, the better. Actions should be possible with as few clicks as possible.
A general guideline to follow when designing the UI is to gently guide users to what they want to do, providing only those actions that are necessary for the current context (or state) of the application. It’s useful to understand the environment where you plan to deploy your application and to understand end user’s goals. Then you can build the user interface and application flow with this in mind.
To do this you define the goals of the application, how long the user is likely to have access to the network (if the application requires network access), and how much time the user can afford for data entry and data lookup. Basically, you tell yourself a story about how you imagine the application will be used and, budgets willing, find out explicitly how a few of the users will use or currently use the application. This is somewhat like a very loosely defined Use case. See the Related Resources section of this article for a list of recommended readings on user interface development and other useful tools.
The sample application for this article, called mAmazon, makes use of most of the concepts discussed earlier. mAmazon lets you compare the price in a bookstore with the price of the same book on Amazon.com.
If Amazon carries the book, the application shows the title and the price. You can drill down to see more information such as the author, the ranking, and the ISBN number.
|Figure 7: Here’s an example of a flowchart.|
You should make a flowchart, like a storyboard, actually drawing the screen as you imagine it would appear in a cell phone. Try to place elements roughly in the order you want them to appear in the final application. Associate the Commands in a separate box, because, as explained earlier, Command placement varies widely from implementation to implementation. You don’t want to assume too much about what the user will see.
Because the MIDP interface metaphor is based on the concept of Screens each element of the flowchart represents an individual screen. Don’t worry too much about reusing the screens or commands at this time; after you’re confident that the model meets the application’s usage demands, you can take the time to isolate commands and determine if it would be advantageous to share commands or other resources in the application.
Amazon is perfect for this application because you can request information in XML from an HTTP query to their server. To do that, sign up as an affiliate and you’re ready to go. Because the MIDP specification demands that all devices provide some form of HTTP communication capability, you’re guaranteed to be able to access the information regardless of the target device. The XML format is convenient because it’s nicely formatted and you can use existing libraries like kXML to parse the data. kXML is an easy-to-use and compact XML parser provided by the Lutris Enhydra Group. You can download it from Enhydra.org.
|Figure 8: The user has very few chioces?either search for a term or exit the search.|
The network aspects of this application are beyond the scope of this article but you should look at the code and use it in your own applications if you find it useful. Also, the application doesn’t take memory constraints into account with respect to data retrieval over the network, but it does page through large data results instead of attempting to display the entire list.
|Figure 9: The search screen while a search is performed.|
To search for a book, the application uses a TextField into which you enter a search term. Navigation is simple; on the search screen the only commands available are to exit or to begin the search.
After beginning a search users can cancel the current search or exit the application. Canceling the search returns the user to the search screen.
|Figure 10: A single search result is returned.|
If the search finds an exact match for the query string then the application displays the detail information screen for that one result immediately, saving users the trouble of clicking the item.
When a search returns more than one result, the application displays the first five results as items in an implicit List. The user can drill down into a result by selecting it, exit the application, conduct a new search, or move to the next or previous set of results.
Finally, when users are on an information screen they see an exit button, a new search button, or, if there are more results, a back button.
|Figure 11: Multiple search results are returned and a scroll button is added.|
With the design reasonably complete, you can start coding UI functionality; the backend development can proceed separately. Start by creating self-contained classes derived from Form. For example, the SearchForm displays a TextField for data entry and prepares commands and titles as appropriate for this form.
Create instances of Commands that you’ve identified as shareable. Make them available in a centralized location. This centralized location will also hold more general Commands such as Exit or New Search. Although you can create a separate class, the sample application handles centralized behavior in the MIDlet’s main class.
In general, forms that contains more than three Commands are potentially confusing to users. The multiple result list screen in this project is an example, of keeping a complicated set of commands down to three (see
To make the user aware of more results than the application can display at one time, the application displays the current result list position (i.e. page 1 of 4) in the Form’s default Ticker item. Placing it in the Ticker guarantees that the information is visible somewhere on the form, and because it iterates it will attract users’ attention.
The network search is a time-consuming process, so a class derived from Runnable performs the search in a separate thread. The doSearch method of the SearchActionForm creates the thread and starts it. A non-interactive gauge tells users how much time is left before the search times out?and the animation shows them that the application is not frozen. Note that a StringItem displays the actual value of the Gauge.
Nothing is more frustrating than being unable to cancel an action, so the application includes a Command to cancel the search. The doCancel method of the SearchActionForm sets the worker thread and the timer task to null, which lets the garbage collector know that it’s OK to release those objects resources and return some memory. The method also sets any unused forms that will not be used to null; otherwise, the VM may keep the resources associated with those forms in memory.
Testing It Out
You should test your applications on as many device emulators as possible. Skins and emulators for most devices are available at little cost or even free. It really pays to test the UI against all possible target devices, following a consistent test path through the UI first, and then trying out unplanned choices. Some devices provide testing facilities. For example, the Palm OS emulator has a great testing facility called Gremlins, which will poke and prod your interface in random ways and report any errors in a log file. Such tools can help you identify problems.
As you can see, it’s not a trivial matter to slap a UI on your application. UI design can make or break your applications. The more devices your application runs on, the more vulnerable it is to UI problems. But you can minimize the potential problems by using the MIDP High-Level UI API and its facilities for creating standard interfaces that render appropriately for disparate devices. Click here to take a look at some general guidelines for minimizing UI problems.
Be sure to separate business rules from the presentation logic so that you can replace UI implementations at compile time. For example, the PDA profile under J2ME provides more facilities for accessing the much more advanced features of higher-end devices such as Palm Pilots and Pocket PCs.
If you have control over application provisioning, you can make different application versions available based on the requesting device’s signature. For example, if your server can tell that an LG5350 is asking for your application, you can serve it a color version. And you can do this automatically without the user having to specify what device they have.
MIDP 2.0 promises to add user interface facilities for specifying spacing, using screen layers and many other exciting and useful additions to our UI toolkit. Take a look at the reference implementation on the Sun J2ME section and check out the examples provided.