Borderless Window Using Areo Snap, Shadow, Minimize Animation, and Shake

Question

I am making a application with a borderless window on Windows. However, since the window is borderless, I have no areo shadow, snap, minimization animation, or shake. I have looked around and found no site that explains how to implement this. However, I know it is possible because Office 2013, Visual Studio 2012, and Steam all have these features and are borderless. I am specifically using QT and C++ but if you have solved this for another windowing library I would like to hear your solutions as well. either. And by areo shadow I don't mean drop shadow on two sides, I mean the glowing shadow on all sides of all active native areo windows applications.

Answer 1

After using Spy++ to inspect Steam's window (its window styles, how it replies to window messages) and trying to match everything it does, combined with the DWMAPI calls from this C# borderless window behavior, I believe I figured it out.

To hide the window's border, handle the WM_NCCALCSIZE message in your WindowProc:

case WM_NCCALCSIZE: {
    if (window->is_borderless) {
        return 0;
    } else {
        return DefWindowProc(hwnd, msg, wparam, lparam);
    }
}

To enable the shadow, all you need to do is:

MARGINS borderless = {1,1,1,1};
DwmExtendFrameIntoClientArea(hwnd, &borderless);

To turn it back off, restore the default margins MARGINS windowed = {0,0,0,0};. Perhaps throw in a SetWindowPos(hwnd, 0, 0, 0, 0, 0, SWP_NOZORDER | SWP_NOOWNERZORDER | SWP_NOACTIVATE | SWP_FRAMECHANGED | SWP_ASYNCWINDOWPOS | SWP_NOSIZE | SWP_NOMOVE ); also, to make sure the frame gets redrawn.

However, this does not seem to work with all window styles, apparently your window style must not contain a titlebar. Title bars work fine, and adding one seems to enable the minimize animation.

The simplest window style I got the shadow to work with was WS_POPUP | WS_THICKFRAME, to also get aero snap, maximizing, minimizing, and the smooth minimize animation I used WS_POPUP | WS_THICKFRAME | WS_MINIMIZEBOX | WS_MAXIMIZEBOX | WS_CAPTION.

Changing DWMWA_NCRENDERING_POLICY or DWMWA_ALLOW_NCPAINT via DwmSetWindowAttribute does not appear to be required, the default settings seem to work.

One word of caution: DwmExtendFrameIntoClientArea does exactly what the name suggests, so if you are drawing an image with an alpha channel directly into your client area (say with opengl, direct3d/2d), a small frame will be visible through it:

So you might have to put a non transparent widget, brush or something behind the transparent element.

If all goes well, it should then look like this:

Here is a small example project, F11 toggles borderless/windowed mode, F12 toggles the borderless shadow on and off.