How painful is it to use your company’s software testing system? If your answer is more painful than dealing with a fender-bender, or coping with an equivalent annoyance, you are not alone.
In fact, you are in the majority, according to a recent survey on software testing done by Osterman Research, and Electric Cloud, a provider of software production management solutions.
Fifty-eight percent of developers who responded to the survey said problems in the testing process or infrastructure -- not design defects -- were the cause of the last major bug found in delivered or deployed software.
When a bug is found in released software, the bottom-line impact on an organization is significant, says Michael Osterman, CEO, Osterman Research.
Fifty-six percent of respondents estimated that their last major software bug resulted in an average of $250,000 in lost revenue and 20 developer-hours to correct, the survey found.
Fifty-six percent also reported that bugs discovered late in development almost always affect release dates.
The survey questioned 144 software development professionals from organizations with at least 1,000 employees and 50 developers across a variety of industries. Most organizations surveyed were located in North America.
“As the software we rely on each day continues to grow in complexity, it becomes more essential that bugs are caught and repaired quickly,” says Osterman.
But most companies don’t do a great job of catching bugs, he noted.
88 percent of the surveyed companies still use manual testing to some extent, and rated dealing with their test systems as more painful than dealing with a fender bender.
“Software developers have a long way to go toward full automation and effective test systems,” Osterman adds.
Developers who felt their companies give enough time to pre-release testing are less impacted by bugs, and spend less than half as much time resolving bugs compared to other developers -- a median of 12 developer-hours compared to 25 developer-hours.
Companies’ continued reliance on slow, resource-intensive manual processes prevents them from being as thorough as necessary in their testing, says Electric Cloud CEO Mike Maciag.
Fully automated test systems save time and use physical, virtual or cloud resources, while greatly reducing the risk of human error, says Maciag.
Automated test systems help make developers and testers more efficient and effective in finding bugs before they reach the end user, add Maciag.
However, the survey shows that completely automated software testing environments are still rare, with just 12 percent of software development organizations using fully automated test systems. Almost 10 percent reported that all testing was done manually.
Other key findings of the survey: 46 percent of developers said they don’t have as much time to test as they should; 36 percent said their companies don’t perform enough pre-release testing; and 53 percent said testing is limited by compute resources.
Electric Cloud provides two tools to help developers automate the creation and testing of software -- ElectricCommander and ElectricAccelerator.
ElectricCommander automates and manages the manual pieces of the build-test-deploy process, making software production faster and more efficient, says Maciag.
The tool is designed to enable developers to implement continuous integration, provide on-demand builds, or automate regularly scheduled build and test cycles.
ElectricAccelerator provides parallel processing for virtually any software production task, and sub-build technology.
“Our goal is to remove the bottlenecks in software production wherever they exist,” says Maciag.