hen deploying a web server and web applications, you must defend against malicious attackers who can identify and exploit the vulnerabilities in these servers and apps. The nimda worm and its predecessors painfully reminded us of that. After all, robust firewall rules and strict router access control lists alone will not protect a web server, which is why web vulnerability scanners are useful tools.

Running a web vulnerability scanner against your web servers will:

• Identify default files and directories that hackers could exploit
• Detect inadequate patch levels
• Point out poor passwords

In this article, I show how scanners achieve this level of defense and how you can utilize them to strengthen the build policy of your web servers. I also offer a review of some of the better known scanners that are currently available.

Anatomy of a Web Vulnerability Scanner
Most Web vulnerability scanners consist of an engine and a database. The database contains a list of directories, file names, CGI scripts, and URLs that have known security risks. Name-your-hat hackers cull Bugtraq postings, vendor advisories, application documentation, or personal favorites to create these lists. The final database usually contains the A, B, and C lists of well-known vulnerabilities, such as the IIS Unicode string exploit (/msadc/..%c0%af..%c0%af..), the Netscape PageServices bug (?wp-html-rend), and /wwwboard/passwd.txt (perhaps running on Apache).

The vulnerabilities can be server-specific like the PageServices bug, which displays a directory listing, or they can be OS-agnostic and target CGI scripts, such as WWWBoard or PHP-Nuke, which expose any server (even Apache) to attacks.

The scanner’s engine is merely a glorified method for making HTTP GET requests for each entry in the vulnerability database. A good engine, however, has some extra techniques for customizing requests. The homebrew crowd, for example, can put together a vulnerability scan using only the echo and nc (netcat) commands (e.g., echo -e “GET /wwwboard/passwd.txt HTTP/1.0

” | nc -vv 80).

Web Vulnerability Scanner in Your Build Policy
Web vulnerability scanners seem to be a favorite tool for malicious users, so why is including one when you deploy a web server so important? The scanner can address the security procedures of your web server build policy, in particular, the following:

1. Testing input validation and removing unnecessary directories
2. Removing unnecessary files and verifying the permissions on the files that rema

in

3. Ensuring strong, secure passwords

Test Input Validation and Remove Unnecessary Directories
Removing all unnecessary directories should be step one of the web server’s build policy. A scanner can assist with this procedure and, to some degree, test a web application’s resistance to input validation attacks. The IIS Unicode exploit is a good example of why this validation is important. A pre-emptive measure against this attack is to map the web document root to a different drive than the system volume (e.g., D:InetPub vs. C:WinntSystem32), which blocks command-line access via vulnerable directories such as /scripts/. Unfortunately, the /msadc/ is commonly mapped to C:Program FilesCommon FilessystemMSADC, which is on the same drive as the system root. So if the sole security measure is re-mapping the web document root, then the server still will be vulnerable because a (very likely) unused directory remains enabled. A good vulnerability scanner will search for the IIS Unicode exploit against default and common directories—of which there can be more than 20.

Remove Unnecessary Files and Verify Permissions
The second step of a web server’s build policy should be to remove unnecessary files and verify the permissions on the files that remain. The majority of checks that a vulnerability scanner conducts relate to sample files, default files, and incorrect file permissions. The default install of a Lotus Domino server, for example, places a slew of databases (.nsf files) within the web document root. Several of these files, especially the names.nsf file, contain sensitive information about the server that any anonymous Internet user may be able to read. You cannot blindly rely on Apache’s security either.

An improper install of the WWWBoard message board application leaves the password file readable to anyone who cares to look for it. Older versions of the Big Brother system-monitoring tool expose arbitrary files outside of the Web document root or they allow command execution. Even improper permissions for .htaccess files give up user passwords.

Ensure Strong Passwords
The hardest part of a build policy to implement (for anything other than HTTP Basic Authentication) is strong password creation. A good scanner will be able to perform rudimentary password guessing to ensure that no passwords can beeasily deciphered. For example, form-based authentication such as mail.yahoo.com or www.hotmail.com is more difficult—but not impossible—to attack.

The Scanner Review
Whisker
Whisker popularized web vulnerability scanning with its Perl implementation, which made extending the URL database easy. In fact, its most under-used capability is its ability to be run as a CGI script simply by placing it in the /cgi-bin/ on your web server. Whisker is best used as a URL scanner. It identifies web pages with known security problems or those pages that should be removed to make a clean web document root. It can also perform brute force attacks against sites using HTTP Basic Authentication.

The disadvantage of Whisker is that it has not been updated in a while, although the author is developing a major update that will add more checks and features. A current version of Whisker also has the capability to scan servers over SSL, but the scanner suffers the drawback of being primarily a URL checker. If it doesn’t find a page, it reports it to the user, but vulnerability checks for recent IIS bugs such as the Unicode or Double Decode directory traversal or Netscape’s PageServices bug are not in this version. They are not easy to implement using Whisker’s current engine, but the following modifications of the /scan.db file serve as a temporary fix:

scan () / > > /?PageServicesevalif( $D{'XXPageSrc'} =~ /index of/i) print "Vulnerable!
";else print "...false alarm ;(
";endeval

The next release, based on libWhisker, will address these bugs.

Stealth
Stealth scanner trades the portability of Whisker for a Windows-style GUI presentation. Stealth is more actively maintained than the 1.x series of Whisker and consequently has a larger database of vulnerabilities. It has Unicode and PageServices checks, but has a high ratio of false positives (reports of vulnerabilities that don’t exist). For example, it may return a false positive for the /?PageServices check if the server always returns a default page. All the scanners share this drawback due to the limited intelligence built into their engines.

Stealth is fast and comprehensive. The user can select a range of IP addresses to scan but cannot input a file list of IP addresses, which would be more helpful for administrators who wish to focus on a Web farm or specific servers.

Nessus
Nessus is a vulnerability checker that does not limit itself to web servers. It takes more effort to set up than Whisker or Stealth does, but it is actively maintained and has up-to-date vulnerability checks. It also returns false positives, however.

Twwwscan/Arirang
The twwwscan/arirang combination is another vulnerability scanner. Twwwscan is a binary program for Windows systems. Arirang is the Unix version, which shares the twwwscan engine. These tools allow the user to specify hosts, networks, and IP address ranges, and to easily customize the CGI checks (through /.uxe text files). Twwwscan checks specific and known server vulnerabilities; but it also has an extensive list of security checks for common misconfigurations that might apply to any homegrown web server. These tools are actively updated.

All of the tools I have discussed share two positive attributes. They have relatively comprehensive lists of vulnerable URLs and they can perform brute force password attacks against HTTP Basic Authentication or rudimentary Form-based authentication. The source code for Whisker, Nessus, and Arirang is available for users who wish to get under the hood and tinker with the engine. The major drawbacks of each engine are the level of false positives and the lack of application-specific checks. The false positives can be reduced with better intelligence when interpreting the results. Stealth and twwwscan run only on Windows platforms, the other tools run on Unix or Windows. BSD users will find that Whisker, Nessus, and Arirang are only a ports update away!

I’ve Scanned My Server. What’s Next?
Application-specific checks are a subset of vulnerability checking that cover input validation problems within the application. The next concern after removing unnecessary files is addressing possible vulnerabilities within the application. These could be attacks that inject SQL statements into data entry fields, embedded script attacks that launch social engineering attacks to collect passwords, or other input validation attacks that lead to arbitrary file retrieval or command execution.

The goal of an application-level scanner is to enumerate all user input fields. These fields can then be catalogued into potential vulnerabilities or functions. Potential vulnerabilities may range from database interaction to OS attacks, while the function of the field could range from login to database entry to search. Defining these categories and creating intelligence to check them is difficult. Whisker, Stealth, and Nessus do not even pretend to perform these types of checks. That’s where Sanctum’s Appscan application vulnerability assessment tool comes in. Its paradigm differs greatly from the file and URL checking of the other scanners, but its price, user interface, and configuration reflect this disparity in level of security.

Now, all you have to worry about is that your Web apps have been coded securely.