Determine Whether a Program Is a Console or GUI Application

Determine Whether a Program Is a Console or GUI Application

When a new process is created, the “subsystem” field in the executable file header determines whether it was linked as a console (also called CUI or “character-based”) or graphical user interface (GUI) application. A console application has a character-mode console window, which it inherits from its parent process upon its creation; if the parent process is not attached to a console, Windows creates a new console and is associates it with the launched application. A newly created GUI application is not attached to a console?even if it is launched from a console application.

The rules above may be modified by passing certain flags to CreateProcess() function: CREATE_NEW_CONSOLE creates a new console for the launched application (console or GUI), while CREATE_NO_WINDOW and DETACHED_PROCESS cause a console application to be launched without a console.

You can parse the header of an executable file to check the subsystem field using the code described in the article “HOWTO: How To Determine Whether an Application Is Console or GUI” in the Microsoft Knowledge Base. Another technique is to use the GetConsoleScreenBufferInfo() function, described in the first method of the article “INFO: Preventing the Console Window from Disappearing.”

The steps of the following method combine several approaches to determine the current program type and the way it was launched:

  1. Get the file name for the current executable using the GetModuleFileName() function with the first argument set to NULL.
  2. Parse the Portable Executable (PE) file header for the executable file name. Use the technique described in the first Microsoft article mentioned above to obtain the way the current application was created (console or GUI).

In most cases, the application type should be known to the developer (it needs to be specified when creating a Visual C++ project; also, the entry point for a GUI application is WinMain()). However, the above steps can be helpful in a library or a class that can be used in either a console or GUI application.

Another method to determine the application type is to use the semi-documented extern “C” int __app_type variable. It’s value is 1 for console applications and 2 for GUI applications. This method will not work for DLL modules.

  • For a GUI application:
         bool bHasConsole = false;     HANDLE hStdout = GetStdHandle(STD_OUTPUT_HANDLE);      if (hStdout != INVALID_HANDLE_VALUE)     {        DWORD dummyMode;        if (GetConsoleMode(hStdout, &dummyMode) )            bHasConsole = true;     }

If the above code sets the value of the bHasConsole flag to true, the GUI application was started with the CREATE_NEW_CONSOLE flag (or AllocConsole() was called prior to this check). Please note that it is also possible to use STD_ERROR_HANDLE instead of STD_OUTPUT_HANDLE for better reliability.

  • For a console application:
        bool bHasConsole = true;    TCHAR lpConsoleTitle[MAX_PATH];    if (!GetConsoleTitle(lpConsoleTitle, MAX_PATH))        bHasConsole = false;    else    {   HANDLE hStdout = GetStdHandle(STD_OUTPUT_HANDLE);         if (hStdout != INVALID_HANDLE_VALUE)        {   DWORD dummyMode;            if (!GetConsoleMode(hStdout, &dummyMode) )                bHasConsole = false;        }    }

If the code above sets the value of the bHasConsole flag to false, the console application was started with the CREATE_NO_WINDOW/DETACHED_PROCESS flag (or FreeConsole() was called prior to this check).

Windows 2000 Professional and later releases provide the function GetConsoleWindow(), which returns “a handle to the window used by the console associated with the calling process or NULL if there is no such associated console.” However, the method above works with any Win32 application.

Share the Post:
Heading photo, Metadata.

What is Metadata?

What is metadata? Well, It’s an odd concept to wrap your head around. Metadata is essentially the secondary layer of data that tracks details about the “regular” data. The regular

XDR solutions

The Benefits of Using XDR Solutions

Cybercriminals constantly adapt their strategies, developing newer, more powerful, and intelligent ways to attack your network. Since security professionals must innovate as well, more conventional endpoint detection solutions have evolved

AI is revolutionizing fraud detection

How AI is Revolutionizing Fraud Detection

Artificial intelligence – commonly known as AI – means a form of technology with multiple uses. As a result, it has become extremely valuable to a number of businesses across

AI innovation

Companies Leading AI Innovation in 2023

Artificial intelligence (AI) has been transforming industries and revolutionizing business operations. AI’s potential to enhance efficiency and productivity has become crucial to many businesses. As we move into 2023, several

data fivetran pricing

Fivetran Pricing Explained

One of the biggest trends of the 21st century is the massive surge in analytics. Analytics is the process of utilizing data to drive future decision-making. With so much of

kubernetes logging

Kubernetes Logging: What You Need to Know

Kubernetes from Google is one of the most popular open-source and free container management solutions made to make managing and deploying applications easier. It has a solid architecture that makes