Editor’s Note: This article discusses a new feature in the Microsoft Visual C++.NET compiler, which at the time of publishing was still in beta. The features described may change before the official release of the product.

uffer overruns are a huge security problem in the software industry, and if you have them in your code you are asking for trouble. If you’re lucky, an attacker will just shut down your application with an access violation. If you’re not so lucky, the attacker can inject and execute malicious code.

There are many ways to help mitigate such threats (see my Best Defense article, “Testing Buffer Overruns”), including education and peer code reviews. Now with the release of Microsoft Visual Studio.NET beta 1, Microsoft has introduced another buffer security tool for the Windows platform: Visual C++.NET’s new compile-time option.

Anatomy of a Buffer Overflow
During a function call on a 32-bit Intel processor, when function A() calls function B(), the CPU must be told to return back to A() when B() completes. When A() is about to call B(), the processor places (pushes) the address of the next instruction after the call to B() onto the stack. When B() completes, it takes (pops) the return address off the stack and continues execution from that address. Any local data B() has also resides on the stack, right before the return address, and if A() passes buffer data (such as a string) to B() then it is also placed on the stack just after the return address.

The following C++ code sample is a function call that I also illustrate in Figure 1, a somewhat simplistic chart of what the stack looks like right after A() calls B():

void B(char *pcBuffer) {    char cTemp[16];    strcpy(cTemp,pcBuffer);}void A() {    B("In a hole in the ground, there lived a Hobbit.");    puts("We just called B()!");}

Figure 1. The Stack Right After Function A() Calls Function B().

Figure 2. The Property Pages Dialog Box.

The danger is if an attacker can overflow the buffer (cTemp) by passing in an overly long pcBuffer, the attacker can overwrite the return address so when B() returns it won’t return to A(). Rather, it will return to the address the attacker just wrote onto the stack as part of the overflow. Specially crafted buffers can make B() return to the start of the cTemp buffer, which may contain malicious assembly language. The possibilities are endless for an attacker.

The best way to solve this kind of problem is to replace calls to strcpy( ) with calls to a somewhat more secure function such as strncat(). I outlined some of these dangerous functions in my Best Defense article “15 Tips for Secure Win32 Programming.”

There is simply no substitute for secure programming practices, but sometimes these aren’t enough. Developers are human, we make mistakes, and we leave vulnerable code in our applications sometimes. Enter the new /GS option in Visual C++.NET.

The /GS Option
The new /GS compile-time option adds special data (a cookie) to the stack between the local data and the return address. The startup code for a process or dynamic link library (DLL) determines and assigns a random value for the cookie. When the function returns (often referred to as function epilog), the compiler checks the cookie; if it has changed, then it calls a default error handler function that halts the process. Stopping the application is better than risking an attack.Setting this option is simple; just follow these steps from within your Visual C++.NET project:

1. Open the project’s Property Pages dialog box (see Figure 2).
2. Click the C/C++ folder.
3. Click the Code Generation property page.
4. Set the Buffer Security Check property to Yes (/GS).

/GS in Action
The following C++ sample code has a buffer overflow—bug—strcpy() copies more than 10 bytes into a 10-byte buffer:

#include "stdafx.h"#include "stdio.h"#include "string.h"void main() {    char szTemp[10];    strcpy(szTemp,"Hello, Big ol' World!");    puts(szTemp);}

When this code is compiled and executed with debug data and the /GS option, you’ll see the dialog box shown in Figure 3.

Figure 3. Buffer Overflow Error Message from Visual Studio Debugger.

Figure 4. Buffer Overflow Error Message from Visual C++ Runtime Library.

It’s actually quite a useful error because it informs you of the variable that clobbered the stack. Clicking the Break button will load the application into the debugger. At this point, you should look at the szTemp variable and find the code and data that led up to the buffer being overflowed.

If the same code is compiled with the /GS switch but without debug data and then executed, you’ll see the dialog box shown in Figure 4.Once you hit OK, the application will halt. You can write your own version of the security error handler function and override the default security handler by calling _set_security_error_handler() early in your application. It may be useful during development to simply call DebugBreak() (or call the x86 assembly language instruction int 3) within the handler to get a clean stack-trace, especially if the application has no user interface like a Windows service.The following code shows how to set a new handler:

#include "stdafx.h"#include "stdio.h"#include "string.h"#include "stdlib.h" __cdecl _myhandler(int iErr, void *pErr) {    if (iErr == _SECERR_BUFFER_OVERRUN)        __asm int 3;}void main() {    _set_security_error_handler(_myhandler);    char szTemp[10];    strcpy(szTemp,"Hello, Big ol' World!");    puts(szTemp);}

Author’s Note: In beta 1, iErr is always _SECERR_BUFFER_OVERRUN and *pErr is always NULL.

For performance reasons, not all functions have the cookie and function epilog code. During compilation, the compiler determines which functions should be protected using two rules:

• The function must be called by another function that passes pointer data.
• The function must have more than eight bytes of stack-allocated buffers.

If you’re wondering how performance will be effected, don’t worry. Microsoft’s internal lab tests on both client and server applications have shown that the overall performance impact is very small, in the order of 1 percent CPU time. Not bad for the extra protection.

The Bottom Line
There is no substitute for good programming skills and peer code review for discovering and fixing vulnerable code. That said, the new /GS option in the new Microsoft Visual C++.NET compiler will help reduce the instances of exploitable buffer overruns in your Windows application code. However, if your application does halt because someone finds a buffer overrun, fix it quickly—and learn from the mistake.