bad response time on your Web site can drive away visitors and prospective customers. Unfortunately, you may not realize that your Web site is having problems until people start complaining. To gain a more quantitative insight as to what your site can handle, you need a way to simulate and analyze different load scenarios.
Using Jmeter, a powerful performance tool from the Apache Jakarta project, you can load test Web pages, Web applications, databases, and several other static and dynamic resources.
First off, download Jmeter. We are using the version 1.9.1 of JMeter on a Microsoft® Windows® XP Professional platform. Unzip the downloaded file into a directory. We’ll refer to this directory as [JMeter-Root].
In order to run JMeter, you need to run one of the following:
- [JMeter-Root]/bin/jmeter.bat (in Windows)
- [JMeter-Root]/bin/jmeter (in Unix)
After the main screen loads, you’ll notice two elements in the left pane, Test Plan and Workbench. The Test Plan is a container for running tests. The Workbench is a sandbox for any test or portion of a test that you are working on. When you are ready to test what you have designed in the Workbench, you can copy or move the elements into the Test Plan.
|Figure 1. Right clicking on the Test Plan reveals the context menu.
The Test Plan
Right-click on the Test Plan icon to see the context menu (shown in Figure 1) that allows you to add sub elements.
You can use the Add feature to create a hierarchy of elements that will then execute when you run your test. The level in which an element exists determines the elements upon which it has an effect. Generally, higher elements within a branch of the hierarchy will affect lower elements within the hierarchy. For example, if you put a timer at the root level of the Test Plan, it will apply to all tests that exist under the Test Plan.
Here’s a summary of the elements to be demonstrated in this article:
A thread group allows you to define the behavior of a group of threads that will interact with your application. Think of each thread as a person sitting at their computer, trying to access a given Web page. You will use thread groups to define the number of threads that a test will launch.
Timers are used to create delays between requests made by each simulated user. Without a timer, your requests hit the server back to back, as fast as possible. Timers allow for a more realistic load on the server. There are different types of timers that allow you to produce either constant or random delays.
Listeners gather data as your tests run, and allow you to report on that data. You can use listeners to create graphs or export data to files.
Target Web Page
This article’s example is deliberately simple. There are two static Web pages: one with an 800×600 pixel image that is approximately 50K in size and another with a 1024×768 pixel image that is approximately 250K in size. Download them here.
Here, two separate tests are conducted to demonstrate the difference in server-side response time. Jmeter can show exactly how the server’s response time differs for a file that is five times larger than another, under the same exact load. This type of test is particularly helpful when deciding whether or not to optimize your Web site images.
Testing Things Out
|Figure 2. Here are the values to set up in the “Small File” Thread Group.
First, make sure that you can load the two static Web pages through your Web browser. Next, create a Thread Group for the smaller file. Right click on the Test Plan and go to Add->Thread Group. The Thread Group should appear under the Test Plan in the left pane. Click on the Thread Group in the left pane and fill out the values as shown in Figure 2.
To test things out, set the Thread Group to run one thread, which represents one user making a request. The ramp-up period specifies the time it takes to reach the maximum number of threads. So, if you had 100 threads and a ramp-up period of ten seconds, Jmeter would launch ten threads per second (100 threads/ten seconds).
Uncheck the Forever checkbox next to Loop Count and set the Loop Count to one for now. The loop count defines the number of times that the thread group will loop before terminating. Bear in mind that the ramp-up period only applies to the first loop.
|Figure 3. This is the HTTP Request Sampler for the “Small File” Thread Group.
Now, you need to tell JMeter where your Web page is. Right click on the Small File thread group you just created and go to Add->Sampler->HTTP Request. Setup the HTTP Request sampler as shown in Figure 3.
The port number and path may differ based on the Web server to which you deployed the HTML files. Make certain that you check the “Retrieve All Images” checkbox under Optional Tasks. This is essential, because youre testing the response time for your pages based on the image sizes.
You want to know if the page has loaded properly, so add a View Results Tree Listener to the Thread Group as well. This listener allows you to see the HTML results for each request that your threads make. Right click on the Thread Group and select Add->Listener->View Results Tree.
To run the test and see if your page pulls up correctly, click on the Run menu at the top and select Start (or just press Ctrl-R).
|Figure 4. View the Results Tree after the tests have run.
Select the View Results Tree and you should see something similar to Figure 4. The image icon in the Response Data area indicates that there is an image in the file. But drill down further under the Request Small File folder—you can retrieve the image there without a problem.
Now that you know your page was pulled up correctly, you need to create a similar Test Group for the large file. A saved version of the test plan generated for this article is available here. Download this file and modify it however you see fit. This saved test plan may not work in previous or future versions of JMeter.
Applying a Load
|Figure 5. Apply a larger load to the thread.
Now that youve verified that you can pull up our files, its time to expand the range covered by the test. Firstly, change the numbers in the Tthread Ggroup. Figure 5shows 50 threads that will ramp-up at five a second for ten seconds. A Loop Count of 12 is also specified—for a total of 600 requests (50 threads X 12 loops).
Right now, JMeter tries to hit the server with as many requests as it can, as fast as it can (after the ramp up period). This load is not very realistic. You can space the threads out a little more by adding a timer. To do this, right click on the thread group and select Add->Timer->Gaussian Random Timer. Set the values to those displayed in Figure 6.
|Figure 6. The Gaussian Random Timer simulates the timing between hits more realistically.
This will delay the threads somewhere between three seconds (the constant delay) and four seconds (the constant delay plus the deviation) per request.
You can view your data in graph form by adding a Graph Results listener. Right click on each of your thread groups and select Add->Listener->Graph Results.
Before you run your tests, clear the data from the last test run. Click on Run in the top Menu and then click on Clear All (or do a Ctrl-E). Also, click on the Test Plan in the left pane and select the “Run each Thread Group separately.” Now, run the tests again and click on the graph results for the Small File Thread Group while they are running.
Looking at the Data
The first thing you should notice is that there is a green square that appears in the upper right corner of the window. This indicates that the tests are running.
Data points should appear on the screen while the tests are running. After running the tests on the Apache Tomcat Web server, on an AMD AthlonTM XP 2000+ 1.66GHZ processor with 512MB of system memory, our collected data was as follows:
Small File (~50K)
Large File (~250K)
So, on average, a file that was five times larger took about 2 times longer to serve under the same load.
A Few Things to Keep in Mind
Make sure you run your performance tests before and after any optimizations are made to determine whether you have in fact made any improvement. If you have a Web application that you are testing and your code changes didn’t make any improvement, revert to what you had originally. Many times, performance optimizations make your code ugly. It is usually not good to hang on to changes that make your code harder to read when those changes didn’t make any significant improvement on your performance.
Another thing to keep in mind is that there are a number of variables that affect the performance of an application. Every little factor from processor speed and amount of RAM to the Web server version can make a huge difference in your performance analysis. Running performance tests in a development environment that differs from your production environment can often be a misleading and misdirected effort.
Another point of concern is network latency, which will occur if you’re testing against a remote server. If you have a staging server that you use for testing purposes, you can run JMeter against that server from a remote network in order to see how network latency might affect your application. You may also want to use the built-in scheduler feature to run your remote tests at different times of the day to account for varying network latency.