When a new process is created, the “subsystem” field in the executable file header determines whether it was linked as a console (also called CUI or “character-based”) or graphical user interface (GUI) application. A console application has a character-mode console window, which it inherits from its parent process upon its creation; if the parent process is not attached to a console, Windows creates a new console and is associates it with the launched application. A newly created GUI application is not attached to a console?even if it is launched from a console application.
The rules above may be modified by passing certain flags to CreateProcess() function: CREATE_NEW_CONSOLE creates a new console for the launched application (console or GUI), while CREATE_NO_WINDOW and DETACHED_PROCESS cause a console application to be launched without a console.
You can parse the header of an executable file to check the subsystem field using the code described in the article “HOWTO: How To Determine Whether an Application Is Console or GUI” in the Microsoft Knowledge Base. Another technique is to use the GetConsoleScreenBufferInfo() function, described in the first method of the article “INFO: Preventing the Console Window from Disappearing.”
The steps of the following method combine several approaches to determine the current program type and the way it was launched:
- Get the file name for the current executable using the GetModuleFileName() function with the first argument set to NULL.
- Parse the Portable Executable (PE) file header for the executable file name. Use the technique described in the first Microsoft article mentioned above to obtain the way the current application was created (console or GUI).
In most cases, the application type should be known to the developer (it needs to be specified when creating a Visual C++ project; also, the entry point for a GUI application is WinMain()). However, the above steps can be helpful in a library or a class that can be used in either a console or GUI application.
Another method to determine the application type is to use the semi-documented extern “C” int __app_type variable. It’s value is 1 for console applications and 2 for GUI applications. This method will not work for DLL modules.
- For a GUI application:
bool bHasConsole = false; HANDLE hStdout = GetStdHandle(STD_OUTPUT_HANDLE); if (hStdout != INVALID_HANDLE_VALUE) { DWORD dummyMode; if (GetConsoleMode(hStdout, &dummyMode) ) bHasConsole = true; }
If the above code sets the value of the bHasConsole flag to true, the GUI application was started with the CREATE_NEW_CONSOLE flag (or AllocConsole() was called prior to this check). Please note that it is also possible to use STD_ERROR_HANDLE instead of STD_OUTPUT_HANDLE for better reliability.
- For a console application:
bool bHasConsole = true; TCHAR lpConsoleTitle[MAX_PATH]; if (!GetConsoleTitle(lpConsoleTitle, MAX_PATH)) bHasConsole = false; else { HANDLE hStdout = GetStdHandle(STD_OUTPUT_HANDLE); if (hStdout != INVALID_HANDLE_VALUE) { DWORD dummyMode; if (!GetConsoleMode(hStdout, &dummyMode) ) bHasConsole = false; } }
If the code above sets the value of the bHasConsole flag to false, the console application was started with the CREATE_NO_WINDOW/DETACHED_PROCESS flag (or FreeConsole() was called prior to this check).
Windows 2000 Professional and later releases provide the function GetConsoleWindow(), which returns “a handle to the window used by the console associated with the calling process or NULL if there is no such associated console.” However, the method above works with any Win32 application.
