Service-oriented architectures (SOA) enable companies to transform their IT business processes into reusable assets by reducing monolithic applications into multiple component services. This composite approach to application development enables businesses to reuse key assets to quickly reconfigure or create new services that better align with the needs of customers, business partners, and users as competitive, economic, and market conditions change. SOA reduces the cost and time of application development or integration and allows businesses to increase their efficiency, developer productivity, and reduce time to market using an agile architectural environment. However, the very qualities that make Web services so attractive to businesses and developers are frustrating quality assurance professionals who are new to SOA testing. Many are only now discovering that the realities of Web services testing are fraught with unexpected—and often daunting—challenges.
As an experienced traditional QA analyst, manager, or engineer you're probably used to testing complete applications. You certainly expect to be given a full build of a working application where everything fits together. However, SOA testing is an entirely different beast. Services are developed and deployed by various developers—sometimes in an ad hoc manner. Instead of a complete application, as a tester you are given a group of services, which may or may not constitute a complete application, and which may or may not have interdependencies on one another. You have to figure out in what order and how they need to be tested as well as discover when an output of a specific service feeds into a subsequent service—or services. Since Web services can be quickly deployed independent of one another, instead of one big release, you have many little releases. You have to test each Web service component as well as its interactions with other services in the SOA application.
In case this isn't enough of a challenge, don't expect to be able to click on a button, type in some data and view the results on a screen because there is no familiar GUI interface. Testing SOA services is essentially equivalent to making a function call. You provide arguments to a function, execute it, and get results back. This requires a more basic and lower level way of thinking than what is required at the traditional application level where data is formatted nicely on a screen and you progress through a predictable and logical sequence of operations. With SOA, you are handed a set of Web services with no guidance and often no obvious path through the services. You have to figure out how to string them together and create a workable, test scenario.
Don't assume you will have access to the Web service as it will eventually be deployed. During development, Web services are likely running on local development machines as opposed to running over a distributed and likely secured network. So testing on a development machine is not the end of the test process. At some point you will have to test in the production environment with a firewall, full security, etc.
Furthermore, when you are testing Web services you are sending to and receiving data from a function. There is nothing preventing you from sending bad data, and no way to know that the data was invalid as opposed to the type of messages and checks you get when testing traditional applications. Since Web service users/consumers have minimal documentation to guide them when they are using the Web service, it is not uncommon for a Web service user to inadvertently send bad data to a Web service. Web services have to be bulletproof so you need to make sure that all types of invalid data are handled properly—no matter how unrealistic the data combinations appear to you. For instance, suppose your service is registering a new user. Typically, a traditional application might provide a drop down list containing legitimate cities when a state is selected. There may be no such filtering in Web services making it possible to enter an invalid city-state combination such as San Francisco, New York. Consequently, expect to run a large volume of tests containing multiple iterations and combinations of data to test a function to ensure the Web service can efficiently handle and process data correctly.
Don't forget about performance. Your Web services may work, but do they scale? Web services can be consumed by anyone for any purpose. Accordingly Web service usage patterns are unknown making it difficulty to predict the volume or load requirements for your SOA application. With no sure fire way to predict usage and load, you must run performance tests to ensure your services' scalability.
Since Web services can be created and changed rapidly you are likely to be inundated when multiple services are sent for testing. Your testing must keep up with this pace. The problem is not the frequency or number of releases as much as the fact that the moment a developer creates or changes one Web service you have to do a full regression test for all the existing Web services in the entire SOA application to test all interactions those new or changed Web services might have with other existing Web services. Manual testing simply can't keep up.
Print
Dzone.com
Digg.com
Slashdot.org
Reddit.com
Del.icio.us