Often developers are tasked with not only developing functional code but also with ensuring that the code they develop scales well and performs in the application environment. The tests that need to be performed against developed code are threefold in nature:

• Function testing: verifies that the code performs its proposed function.
• Scalability testing: verifies that the code consumes as few resources as possible while performing its proposed function.
• Goal testing: verifies that the code executes in less time than any specified service level agreement (SLA).

Of the three tests, usually function testing is the easiest of the three to verify.

This article covers the difference between scalability and goal testing, illustrates an example of pseudo-code test harnesses for manual testing, and demonstrates an example test of an Oracle procedure using an automated testing interface (Quest Software?s Toad). This article provides direction for developers seeking to learn about proper code testing and the tools available to make testing accessible to developers.

The Challenges of Manual Scalability Testing

The first type of test to verify that a program, function, or statement uses the fewest possible resources is called a scalability test. In a scalability test the code is placed into an environment similar to what is expected in the production system and then it is run with random variables and a steadily increasing user load. The user load is increased either to a specific number of users, or, the number of users is increased until stress is seen in the environment to determine the ultimate scalability on the existing platform.

Scalability testing can be performed using an in-house generated test harness similar to the pseudo-code in Listing 1. Of course, you have to run the test script manually from several users at the same time.

Maintaining the test harnesses for perhaps hundreds of statements is a daunting task. In addition, generating the random values and maintaining the underlying infrastructure for manual test harnesses also would require time and effort better put into maintaining and improving the actual application.

The results from scalability tests are usually specified in transaction per second (TPS) or transaction per minute (TPM).

Goal Testing

The other aspect of code testing for performance is testing to see that code executes in less time than any specified SLA; both as a single statement execution and when run in concert with multiple users. When committing to an SLA, the management team needs to commit to only those sections of the system they control.

A good SLA is, “Transaction X will complete within Y milliseconds (or seconds) at the database level with Z number of simultaneous users.” A poorly worded SLA example is, “Screen X will complete within Y seconds.” Why is the second example a poor SLA? Because the user seeing screen X could be next to the server and chances are good that the SLA can be met. On the other hand, the user could be in Bangladesh while the server is in the U.S. In this case, all bets are off in meeting the SLA once multiple network latencies start coming into the picture.

An example test harness for manual goal testing is shown in Listing 2.As with the scalability test, the manual test needs to run simultaneously from multiple users to fully test the SLA. Goal test results are usually reported as TPM or TPS required too reach a specific number of seconds or the number of users required to reach a pre-specified TPS or TPM.

Automated Testing for Scalability and Goal Testing

Several companies provide utilities that have automated testing capabilities, for example, Load Runner from HP (formally from Mercury) provides a rich testing environment. However, it may be overkill for the single developer wishing to perform scalability and goal testing on a single statement, procedure, or functional component. Toad for Developers from Quest Software provides a more single-user friendly test environment for the developer.

An Automated Scalability Test Example

The following example looks at an actual scalability test, which uses Toad for Oracle. The first step in the cycle is to develop the required code. In this case, it is a procedure that inserts a record into an employee table. Figure 1 shows this procedure in the Toad editor.

Before creating the procedure, I defined a set of tests in Toad Code Tester. I allowed it to generate a test harness to automate functional testing. Figure 2 shows the results of a Code Tester run against the compiled procedure from Figure 1.

Figure 1. Example Procedure in Toad. The first step in the cycle is to develop code that inserts a record into an employee table.

?

Figure 2. Results of a Code Tester. This figure shows the results of a code tester run against the compiled procedure from Figure 1.

Scalability and/or goal testing

To perform automated testing the automated link is utilized in the Benchmark Factory interface. Then, the next step is to choose a PL/SQL procedure. The scenario wizard for procedure testing from Benchmark Factory (BMF) is shown in Figure 3 with the chosen procedure selected. Following the selection of the procedure to be tested, the wizard allows you to insert calls to provided randomization scripts to ensure random variables are utilized during the testing of the procedure (see Figure 4).

Figure 3. Procedure Selection. This figure shows the scenario wizard for procedure testing from Benchmark Factory.

?

Figure 4. Choosing Variables. The wizard allows you to insert calls to ensure random variables are utilized.

Next Step

Choose the range of users used to execute the code simultaneously. In BMF, simply supply a lower and an upper bound and the increment level. Figure 5 shows a test range of 1 to 50 with an increment of 10.

The test is executed after the user load profile is set. The results are provided in both tabular and graphical format and you can dump it into an Excel Spreadsheet. Figure 6 shows the first page of the tabular output from the procedure test.

Figure 5. Setting the User Loads. This figure shows a test range of 1 to 50 with an increment of 10.

?

Figure 6. Tabular Output. This figure shows the first page of the tabular output.

Of course, management likes graphs, so who are you to argue? Figure 7 shows one of several graphs that you can output from the tool. The spreadsheet output also contains multiple graphs ready to be cut and pasted into manually written reports if needed.

	?
Figure 7. Example Graphical Output. This figures shows one of several graphs that you can output from the tool.

The benefits of an integrated development, functional test, and scalability/goal test environment at the individual user level should be apparent. By using standard tools each developer produces code and testing reports for all phases of testing that are passed on to the QA/QC group when they perform integration testing. To help accomplish your goals, discovery phase services study your project details to recommend a plan to fit your requirements.

Automated Code Testing for the Developer

This article covered the difference between scalability and goal testing, showed example pseudo-code test harnesses for manual testing, and showed an example test of an Oracle procedure using an automated testing interface. This article should provide direction for developers seeking to learn about proper code testing and the tools available to make testing accessible to developers.

Featured image provided by Kevin Ku; Pexels; Thanks!