With mobile application capabilities and usage increasing, the application functions that are performed on a mobile device are also becoming more complex in functionality and more critical to businesses. The mobile mode of service delivery -- being closest to end users and always available -- requires that developers pay particular attention to mobile application performance testing. Even users expect more responsive performance from mobile applications than they do from desktop apps.
So, it has become extremely important to determine the performance that a mobile application's end-user will experience before actually deploying the application onto a mobile platform. To do so, mobile application developers need to performance test their applications with an end-to-end application delivery model that tests for all the varying scenarios on each component involved.
In this article, we propose such an end-to-end approach to mobile application performance testing. Our aim is to present a performance testing exercise that will help mobile application developers accurately gauge performance from the end-user's point of view.
Mobile Application Testing by Type
It is important first to understand the types of mobile applications developed today. Each type calls for a tailored performance testing approach targeting the components and processes involved.
Testing Browser-Based Mobile Apps
Browser-based mobile applications might perform differently as compared with the same application accessed via a desktop browser. The causes of these differences include the bandwidth limitations of the user's cellular network data plan and differing device configurations. The advantage of a mobile browser-based application is the lower development cost, as the mobile app developer needs to consider only how to make it compatible with most mobile browsers.
While testing for performance, it is important to replicate the user server load from mobile browsers too, in case additional dedicated components are being deployed to cater to such requests. It is also important to test Web page rendering on the target device as device configuration (e.g. CPU, memory or browser rendering engine) can affect end-user performance.
Testing Native Mobile Apps
Native mobile applications are installed directly onto mobile devices, allowing users to access the applications quickly. The disadvantage of native mobile apps is the development cost. Because the applications are platform dependent, an app developed for one mobile OS won't work for another. The developer is left having to develop and test the application on all the platforms, such as iOS, Android, Blackberry, and so on. This calls for thorough performance testing on each target platform as well as on a set of devices in each selected platform.
Testing Hybrid Mobile Apps
Hybrid mobile applications are installed on mobile devices just like native mobile apps but users can access them via either an installed app or the device browser. The application is developed using Web technologies and then wrapped in a platform-specific shell that allows the app to be installed just like a native app.
As with native applications, the disadvantage of hybrid mobile applications is development cost. To allow a user to access the application via an installed app, the developer has to develop the application for all the platforms mentioned in the previous section. Again, a performance test strategy has to target the load generated by users of such hybrid apps as well as gauge the on-device application performance.
Mobile Application Performance Drivers
An ideal mobile application performance test should target every performance driver. Figure 1 presents the key components governing a mobile application's performance.
Click here for larger image
Figure 1. Key Components Governing Mobile App Performance
For a typical application delivered to a mobile device, the performance as perceived by the end user depends on two key factors:
- Content delivery to device: This process involves the request delivery to the server, response generation at the server, and the response delivery back to the source device. Typical parameters that affect this are load on the server components, type of network involved, load on the network, and the external component behavior. These parameters can be handled and tested for by emulating the expected real world conditions on the server and network.
- Content rendering at device: This process involves displaying the response from the server at the source device. Typical parameters that affect this are type of platform, type and configuration of device, and the single user behavior patterns. These parameters can be handled and tested for by in-parallel, on-device application access and monitoring.