Tackle Device Fragmentation with NetBeans and the NetBeans Mobility Pack

ince its inception, “write once, run anywhere!” has been the mantra and rallying cry for many architects and software engineers trying to convince their enterprises to move to or stay with Java. The phrase was particularly useful when talking about building software applications for mobile and wireless devices.

If you are involved in J2ME development, the write once, run anywhere or “WORA” battle cry may have lacked some luster in recent years. Sure it sounds good and logically makes sense. Java was extremely successful in assisting developers write software applications that run on a number of operating systems (Windows, Unix, Linux, Mac OS, etc.). It would seem to follow that Java is also suited to help support application development on the multitudes of cell phones, personal digital assistants (PDAs), and other consumer electronic and embedded devices. Unfortunately, it turns out the diversity of devices is too much for J2ME and perhaps any software platform to handle. If you have tried to write an application that runs on a couple of types of cell phones (let alone trying to cover the millions of J2ME-enabled phones or other electronic devices) you are probably dealing with this problem. It has a name and it is called device fragmentation.

Defining Device Fragmentation
Device fragmentation is the problem of having to write separate or custom software for devices on which you want your application to execute. In other words, it is writing and managing a separate code base for the same basic functionality just to meet individual device capabilities/features. The worst-case scenario is being forced to write a separate code base for each and every device you what to support?certainly a WORA kill-joy. Device fragmentation turns the idea of creating and mass distributing that “killer app” you have in your head and want to develop into an almost insurmountable problem.

What does the problem stem from? There are many reasons why devices do not avail themselves to WORA. Some of the problems relate to device capabilities such as screen size, available memory, storage space for your code, and processor speeds. Other problems relate to the diversity and capabilities of the underlying operating system of the devices. Finally, the choice of J2ME profile/configuration (example: MIDP 1.0 vs. MIDP 1.1) and JVM, adherence to the standards for that JVM, and availability and implementation of the various J2ME APIs (like the Mobile Media API or J2ME Web Services) makes for many other obstacles that must be overcome to provide true application portability.

Device Spectrum
In order to give those that have not had to address device fragmentation a taste of these issues, simply go to Sun’s listing of J2ME supported devices. First, take a look at the various screen sizes supported by the devices. You will see everything from 95×65 to 640×200 pixel resolution and one to eighteen bits of color depth. Imagine trying to write a J2ME game application given that type of variety in user interfaces? Next, take a look at the “Software” column listing the J2ME profile, configuration, and J2ME optional APIs that each device supports. Remember that a simple change from CLDC 1.0 to CLDC 1.1 in J2ME means the difference between having something as simple float point numbers or not. By the way, a vendor’s interpretation of J2ME standards like MIDP means that running an application on devices that each support MIDP to the 1.1 level, for example, does not guarantee consistent behavior. Finally, take note of the number of manufactures and models of devices?over 170 different devices in this list and many with their own OS and unique device capabilities. Nokia (the world’s number one manufacturer of mobile phones) alone has two major operating systems (Nokia and Symbian) on the J2ME phones listed. Can you imagine your test plan to check all the features of your application against a major chunk of these devices? Ouch!

Some Defragmentation Help
Not a pretty situation, but you have to at least acknowledge that Java and J2ME have offered hope to hopelessness. The JVM on millions of phones provides the basis for running a large portion?probably 80 percent in most situations?of an application without issue. And before you are ready to abandon J2ME and decry WORA dead, know that there are some tools that help. Specifically, using the NetBeans IDE in conjunction with the NetBeans Mobility Pack can help you address some of these device fragmentation issues.

Exploring NetBeans Project Configurations

Figure 1. Opening the StockQuote Project: After unzipping the contents of the zip file associated with this article, bring the StockQuote project into NetBeans by using the Open Project option under the File menu.

If you have not installed or used the NetBeans IDE, please refer to this article, posted on DevX this summer, in order to get started with NetBeans and the NetBeans Mobility Pack. Once familiar with NetBeans and the Mobility Pack, the code associated with this article and information below can help you write an application to reduce your device fragmentation issues. The operative word here is “reduce,” not eliminate?more on this later.

The application provided with this article is a simple stock and mutual fund price quote J2ME MIDlet suite. It will help demonstrate the ability of NetBeans and the Mobility Pack to provide some device fragmentation relief. To get the application working, download and unzip the contents of the source code file onto your file system. It will create a StockQuote folder wherever it is unzipped. Next, start NetBeans and use the “Open Project” feature in NetBeans (as shown in Figure 1) to locate the StockQuote folder and import the project into the IDE.

Examine the project in NetBeans and you should find one of the MIDlets in the suite is RequestQuoteMidlet. This midlet obtains the latest stock or mutual fund price from a financial Web site and stores it into a J2ME record store (see Figure 2).

The second MIDlet in the suite, HistoricQuotesMidlet (see Figure 3), displays the previously stored price quotes for a stock or fund.


Figure 2. RequestQuoteMidlet: RequestQuoteMidlet allows users to enter a stock or mutual fund symbol and obtain, via wireless connection to the Web, its current price which it stores in a record store for later retrieval.
 
Figure 3. HistoricQuotesMidlet: Using the HistoricQuotesMidlet, previously obtained stock or mutual fund prices, obtained through RequestQuoteMidlet, are displayed in a chart.

Simple enough, but let’s assume you want to make this application available to three J2ME-enabled mobile devices. The devices share some features and characteristics, but each offers fragmentation issues. Table 1 below specifies the major differences of the devices and how they relate to the application.

Device A – older J2ME Phone

Device B – new J2ME Phone

Device C – PDA

Supports MIDP 1.0/CLDC 1.0 and therefore cannot use floats for price information.

Supports MIDP 1.1/CLDC 1.1. Can use floats for prices.

Supports MIDP 1.1/CLDC 1.1. Can use floats for prices.

Smaller screen and able to display only a limited number of quotes.

Also has a smaller screen to display only a limited number of quotes.

Larger screen to display more quotes and graphics.

Has no sound to audibly inform users when a stock quote has been retrieved from the Web.

Has sound (MMAPI support) to provide audible indication to users that a price has been retrieved.

Has no sound to audibly inform users when a stock quote has been retrieved from the Web.

Does not require obfuscation.

Does not require obfuscation.

Obfuscation required. The class files on the device are more accessible and therefore vulnerable to reverse engineering, etc.

Table 1. Stock Quote Device Fragmentation Issues: Even with a simple application such as that represented in the StockQuotes MIDlet suite, a number of device fragmentation issues as shown in this table can arise.

NetBeans Project Configurations
In order to address some of the device configuration issues, NetBeans and the Mobility Pack provide a feature called “project configurations.” Essentially, a project configuration defines a device environment in which your application will execute. In general, a JAR and JAD file are created from each project configuration. A single set of source code is maintained, but the project configurations provide direction to the IDE on how to build and generate the deployable code files for the various environments?stripping out and adding elements as necessary.


Figure 4. MIDP Properties Dialog: Edit the properties associated with a project configuration through this dialog box. To create or remove a project configuration, hit the manage configurations button.
 
Figure 5. The NetBeans with Mobility Pack IDE Button Bar: The drop down list in the middle of the IDE’s button bar sets the current project configuration.

Several project configurations can be and often are defined for each project (or MIDlet suite). A project configuration defines such elements as what version of the MIDP profile and CLDC configuration a target platform provides and what optional J2ME APIs are installed, thereby restricting application code accordingly. It also specifies what emulator to use in order to test the application. Other environmental properties include a project configuration file that defines include API permissions, obfuscation levels, and other build parameters, security domains and signing, adjustments to the application descriptor, and “abilities” which are identifiers that can be used to associate source code to one or more specific features of a target device like having a larger screen or sound.

To create or manage a project configuration, right click on the NetBeans MIDP project and select “Properties” from the menu. In the ensuing dialog window (see Figure 4), a drop down list of existing project configurations and a “Manage Configurations…” button allow you add and remove project configurations. For StockQuote, three project configurations have already been created (OldPhone, NewPhone, and PDA) to represent the three target environments listed in Table 1 along with the DefaultConfiguration, which is provided by default with any new J2ME project. Select any of the project configurations and then take a look at the Categories listed on the left side of the dialog window. Click on the categories and examine the configuration properties that can be set in a project configuration and you begin to see how a tool like NetBeans can help with some of the fragmentation issues.

You will also notice on the NetBeans IDE button bar a drop down list that contains the names of the project configurations (see Figure 5). This is your means to quickly select and set the current project configuration as you work on your application. Based on which project configuration is selected, the appropriate code is built and emulator used in testing your project.

Abilities
Click on the category called “Abilities” in the Properties dialog (as shown in Figure 4). As already noted, abilities are just identifiers or tags that are associated with a project configuration. Abilities suggest what specific feature or “ability” the target device might have. In the provided example code, three abilities have been created: bigScreen, hasSound, and newJ2ME. These abilities have been associated with the project configurations as shown in Table 2 and, as one might expect, given the capabilities of the devices suggested in Table 1. For example, the new phones are capable of playing sound, so the hasSound ability is associated to the NewPhone project configuration.

Device A – OldPhone project configuration

Device B – NewPhone project configuration

Device C – PDA project configuration

no abilities

hasSound

bigScreen

 

newJ2ME

newJ2ME

Table 2. Project Configurations and Associated Abilities: Abilities are used to tag or identify device capabilities or characteristics of a target device specified by the project configurations.

Alone, abilities and their associations to project configurations really don’t mean much. Its in their associations to lines of source code in preprocessor blocks (see the next section) that they allow you to tackle device fragmentation issues associated with device “abilities” like sound and screen size.

Preprocessor Blocks
Through special comment markups, lines of source code can be associated with project configurations and/or abilities. Known as “preprocessor blocks,” these markups in the code, associating the code with either a project configuration or ability, indicate to the build facility in the IDE whether to add or remove the code when building the application class files. In other words, they help to define code that is active only for a specific device or device feature.

Figure 6. Managing Preprocessor Blocks: To set, unset, or modify a preprocessor block around any Java code in your MIDP application, select and highlight the code for the block and right click. Then select the appropriate project configuration or ability to associate with the block.

A small code example will help explain and demonstrate preprocessor blocks. Suppose, based on the size of a device’s screen, you wanted to limit the size of a text field. Using preprocessor blocks and project configurations in NetBeans, the source code might look something like the following:

/*#BigDevice#*/entryField = new TextField(“Your name:”,null,25,TextField.ANY);/*$BigDevice$*//*!#BigDevice#*/entryField = new TextField(“Your name:”,null,10,TextField.ANY);/*$!BigDevice$*/

The preprocessor comments provide a kind of if/else conditional creating the appropriately sized text field (either 25 or 10 characters long) as needed depending on whether the code was compiled for the project configuration defined as BigDevice or not BigDevice (!BigDevice).

And how do these special preprocessor blocks get placed in the code? NetBeans Mobility pack provides a special code editor feature that allows any code to be easily and quickly associated with a project configuration and/or ability. Simply select text in the code editor and right click. A menu will appear and at the top of the menu (as shown in Figure 6) is an option for Preprocessor Blocks. Clicking on the Preprocessor Blocks option reveals a list of your project configurations and abilities. Simply select a project configuration or ability to put the code in a preprocessor block. The IDE will then take charge of putting the appropriate commenting around the code that the compiler will pick up and use when building your project. You will also notice that an “inverse” or “else” conditional preprocessor block can be set up using the same menu options. NetBeans will also use the same comments to appropriately highlight and comment/uncomment the preprocessor blocks while you are working on the application based on whichever project configuration is selected. This allows you to work with one source code file set and, through the selection of project configuration, drive the inclusion or exclusion of code blocks from the application that is built, tested, and ultimately deployed.

Applying Project Configurations, Abilities, and Preprocessor Blocks
To get a better understanding of project configurations, abilities, and preprocessor blocks, you can use these NetBeans Mobility Pack features to address the device fragmentation issues associated with the StockQuote MIDlet suite.

Figure 7. Multiple Build and Dist to Support Project Configurations: The NetBeans IDE with Mobility Pack uses multiple build and distribution folders in a single MIDP project to manage the multiple sets of class files and deployment code it creates based on the project configurations. You will see one build and dist folder for each project configuration associated with your project.

New vs. Old J2ME
First, let’s take on the issue that two of the targeted devices support newer versions of MIDP and CLDC while another of the target devices, namely an older J2ME phone, does not. This is a common problem in J2ME development and its greatest impact in this application is what primitive types each supports. With CLDC 1.1, floating point numbers were added?an important feature when trying to obtain, store, and retrieve dollar and cent amounts that are often expressed in floats. For the older phone, two separate integer values must be managed to track dollars and cents. This problem could easily necessitate having to write two separate applications.

Using an ability labeled newJ2ME and preprocessor blocks, program lines dealing with both integers and floating point numbers can be added to the same source code. The IDE editor and compiler will use the preprocessor blocks marked by the ability labels to essentially comment and uncomment various lines of the application based on which project configuration (and thus, which associated device) is selected. For example, the following code in the HistoricQuotesMidlet retrieves prices out of the record store. Depending on which project configuration with its associated ability (in this case whether a newJ2ME device) is selected, the corresponding preprocessor blocks allow for prices to be retrieved as an array of floats in one case (newJ2ME ability) or as an array of integers in the other case (!newJ2ME ability?meaning not having the newJ2ME ability). In the case below, a project configuration with newJ2ME ability is selected as the !newJ2ME ability code has been commented out by the IDE.

    /*#!newJ2ME#*/////--    int[] prices = retrievePrices(currentSymbol);    /*$!newJ2ME$*///    /*#newJ2ME#*///    float[] prices = retrievePrices(currentSymbol);    /*$newJ2ME$*

These preprocessor blocks are sprinkled throughout the StockQuotes example code. Yes, it does mean that you have to write code for each instance of device fragmentation, but at least it’s all in a single code base and the IDE helps you assign and modify associations of code with particular devices. In fact, the way the IDE does this is that it actually manages multiple mini-projects for your single code base. Open the files tab of the NetBeans IDE as shown in Figure 7 and you will notice that there is a build and distribution folder for each of the project configurations. These are all managed by NetBeans, but you can see how your project configurations lead to this file structure.

Figure 8. Big vs. Small Screen Device Fragmentation: Using an ability and preprocessor blocks, screen information appropriate to the project configuration and associated devices is turned on or off to allow the application to generically build these displays without worrying about the screen sizes.

Screen Size
Abilities and preprocessor blocks can also be used to deal with the various screen display sizes. The PDA device has a much larger display screen. Therefore, in this example, the chart displaying stock and mutual fund prices should be scaled to take advantage of larger display. Furthermore, more historical prices can be displayed when a larger screen is available (see Figure 8).

In the ChartCanvas code that graphically displays the stock quote prices, a second ability is used to designate certain screen parameters for “big screens” versus non-big screens. The application then uses these parameters to effectively scale the drawing to the size of the screen associated with the project configuration’s target devices.

  /*#bigScreen#*/////--  static final int MAX_PRICES_DISPLAYED = 10;//--	static final int START_X_POSITION = 60;//--  static final int MAX_BAR_SIZE = 500;//--  static final int ROW_HEIGHT = 20;  /*$bigScreen$*///  /*#!bigScreen#*///  static final int MAX_PRICES_DISPLAYED = 5;  static final int START_X_POSITION = 40;  static final int MAX_BAR_SIZE = 100;  static final int ROW_HEIGHT = 15;  /*$!bigScreen$*/

Additional APIs
Some devices may have special features and optional J2ME APIs installed on the device to use the special features. For instance, in this example, the new J2ME phone devices that the NewPhone project configuration targets can play sound files (WAV, MIDI, etc) using the optional J2ME multi-media API (MMAPI). This is another case where a separate application code base would often be needed in order to use this API. Instead, more ability related preprocessor blocks and a special project configuration property can be set to allow the inclusion and use of the MMAPI without the need for a separate application.

In the code, when a new commodity price has been requested, receiving the price quote reply from is asynchronous. Therefore, when the price is received, it is nice to provide the user with an audible signal (in addition to displaying the price information on the screen) so they know when the price has been displayed. In the RequestQuoteMidlet, you will find the preprocessor blocks that include a call to the MMAPI to play a .wav file when sound is available. In other words, when the hasSound ability is associated with the project configuration.

/*#!hasSound#*/////--      Alert symbolAlert = new Alert("Check Symbol/Type",//--          "No quote found.", null, AlertType.WARNING);/*$!hasSound$*////*#hasSound#*///      Alert symbolAlert = new Alert("Check Symbol/Type",          "No quote found.", null, null);           try {		InputStream is = getClass().getResourceAsStream("/audio/bong.wav");		Player audioPlayer = Manager.createPlayer(is, "audio/x-wav");                audioPlayer.start();      } catch (IOException ioe) {      } catch (MediaException me) {      }/*$hasSound$

As a second step in dealing with this device fragmentation issue, for the NewPhone project configuration must also designate the inclusion of Mobile Media API as part of the target platform. Switch to the NewPhone project configuration using the button bar project configuration drop down list. Open the project properties dialog window and click on the Platform category (see Figure 9). Note that for the NewPhone configuration, the Optional Package Mobile Media API check box is switched on while it is off for other configurations.

Figure 9. Switching on Optional APIs in the Project Configuration:The project configuration also allows you to designate the associated devices with having or not having various optional APIs?like MMAPI?available to the application.

In order to build and test this portion of the application that uses the MMAPI to play sound, you will have to have the MMAPI available to the IDE and your application. Download and install the Sun reference implementation of this API. In this example, MMAPI has been installed at c:mmapi-1.0-fcs and the code downloaded with this article references the MMAPI and sound files from that location. If you change the location, be sure to also change the references to the API and associated resources in the project properties of the project configuration that uses the API. Namely, switch to the NewPhone project configuration and change the MMAPI reference listed in the Build/Libraries and Resources for the project configuration to point to wherever you have the MMAPI library installed.

Obfuscation Changes Per Device
Obfuscation modifies bytes code during builds to make it harder to decompile or reverse engineer your application (see this article). Reverse engineering might be more of a threat on some devices, like PDAs, where access to the file system is particularly easy. As the requirements table (see Table 1) suggests, the code deployed to the PDA device ought to be obfuscated in order to better protect the intellectual property within the application. Here again, NetBeans project configurations can help. Switch to the PDA project configuration using the button bar project configuration drop down list. Open the project properties dialog (right click on the project and select Properties) and click on the Build/Obfuscating category in the project properties dialog window. For the old and new phone project configurations, the obfuscation level is left off, but for the PDA project configuration, the level is set to high. Whenever the PDA project is built, the appropriate obfuscation level is applied to the class files while not affecting the class files created for the other projects.

Some Glitches/Issues
While NetBeans’ use of project configurations, abilities, and preprocessor blocks to address device fragmentation is slick, these elements do add a few glitches to your programming. Namely, I noticed that references to line numbers in stack traces are thrown off by the inclusion/exclusion of the various preprocessor blocks; presumably because some lines of code are removed or added based on the preprocessing. This makes testing and debugging applications a bit more difficult. Secondly, using NetBeans’ reformatting tool to clean up and align your code is also thrown off by the special tags and comments associated with the preprocessor blocks. Finally, the NetBeans Mobility Pack was established to help write MIDP/CLDC applications. Therefore, if you are building other types of J2ME applications, say a CDC application for example, you will not find NetBeans and NetBeans Mobility Pack project configurations very helpful.

Alternative Solutions and Work to Do
NetBeans is not the only tool available to address the device fragmentation issue. J2ME Polish is a an open source suite of tools to help, as their Web site states, “polish” J2ME applications. Along with tools to obfuscate and optimize your application, J2ME Polish provides several tools for wrestling with device fragmentation issues. Also, Tira Wireless offers a commercial product, the Tira Jump Product Suite, to address device fragmentation. In fact, Tira Wireless’ CTO and co-founder is Allen Lau. Mr. Lau has dedicated a large part of his professional career on addressing the device fragmentation issue. A simple search for device fragmentation on Google or your favorite Web search engine will undoubtedly connect you to several articles he has written or statements he has made on the subject.

When I asked him via email if he was satisfied with the direction open source tools like NetBeans and J2ME Polish take in addressing the device fragmentation problem, his response was definitive. “No. Both assume that API fragmentation is the root cause of the problem while the real problem is implementation fragmentation.” He explains: “the most serious problem is implementation differences?screen size, available heap memory, performance, JAR size limitations, and, most importantly, VM implementation differences” which he claims NetBeans and J2ME Polish have not gone far enough yet in their efforts to address. “Standardization through JSRs,” he claims, “will not be able to solve [implementation differences]. JSRs can only resolve API fragmentation issues which is not a big problem today.” Solving the big problem of implementation differences “is out of the scope of the JCP and in fact is impossible to standardize. A mobile phone is a consumer electronic. Manufacturers have to differentiate themselves and segment the market (e.g. low end vs. high end devices). As such, it is actually not desirable to standardize. Using games as an example, people in the console game business accept the fact that they will have to develop for PS2, Xbox, and GameCube. It is just part of the development cost. We, as professionals in the mobile industry should also accept the fact that the cost of device fragmentation will never reduce to zero.”

Indeed, “solving” device fragmentation might be impossible. Java/J2ME provide a foundation for WORA, but complete and seamless portability in the device world is tough. Tools to help address it without having to write and manage separate code bases are emerging. Open source tools like NetBeans/NetBeans Mobility Pack and J2ME Polish offer some first steps and, as Allen Lau states, using tools like his Jump Platform help keep “the ‘cost’ of fragmentation down to a very reasonable and manageable level that fragmentation is no longer unbearable.”

Share the Post:
Share on facebook
Share on twitter
Share on linkedin

Overview

Recent Articles: