Win32: How to make drop shadow honor non-rectangular Layered window?

Question

i've created a layered window by adding the the WS_EX_LAYERED extended style:

wndClass.ExStyle = wndClass.ExStyle | WS_EX_LAYERED;

Windows will use black as the chroma key color value. i'm going to leave a large border of black to make the problem obvious:

---

After the window is constructed, i tell it to use black as a chroma-key color:

SetLayeredWindowAttributes(hwnd, 0x00000000, 255, LWA_COLORKEY);

Now the popup layered window appears partially transparent:

---

The problem is the final step. i want to use CS_DROPSHADOW class style, available since Windows XP, to create a drop-shadow:

wndClass.Style = wndClass.Style | CS_DROPSHADOW;

The drop shadow appears, but the shadow surrounds the original rectangular window, and doesn't take into account the window's transparency provided by the layered window:

Does anyone know what magical option i've missed somewhere that will make the drop shadow honor the non-rectangular layered window?

---

Another example of where this issue appears is when you don't include the 6px padding/margin. The hint window as drawn by Windows® themes is non-rectangular. This leaves a small visible gap where the window is transparent, but the drop shadow does not appear:

Microsoft has managed to make it work, as you can see from this hint from Internet Explorer:

---

Looking closer at a Windows tooltips class hint window. Using SpyXX - i can get its window rect, and class styles:

SpyXX says:

Rectangle:     (440, 229)-(544, 249), 104x20
Restored Rect: (440, 229)-(544, 249), 104x20
Client Rect:   (0, 0)-(104, 20), 104x20

So everything points to the window itself being 104x20 pixels, with the drop shadow outside the window itself. (Which is consistent with CS_DROPSHADOW.)

Next i can look at the styles of the tooltips window class:

Windows Styles:     94000001

    WS_POPUP        80000000
    WS_VISIBLE      10000000
    WS_CLIPSIBLINGS  4000000
    TTS_ALWAYSTIP          1

Extended Styles:    00080088

    WS_EX_LAYERED      80000
    WS_EX_TOOLWIN         80
    WS_EX_TOPMOST          8

Interestingly, it doesn't use CS_SAVEBITS (0x800); which is useful for small, short-lived, windows.

Nor does it use CS_DROPSHADOW (0x20000). So now i wonder how is it drawing outside its own window?

Note: Transparent layered windows is documented as the preferred technique over regions.

Edit: Layered Windows have been around with Windows 2000. CS_DropShadow was added with XP.

Answer 1

Transparent layered windows is documented as the preferred technique over regions.

However, CS_DROPSHADOW does pay attention to regions. If you crop or otherwise shape your window using a region, the drop-shadow will follow the new outline.

Fortunately, you can use regions with layered windows, and by combining the two get the effect you're looking for.

BTW: tooltips_class32 does use CS_DROPSHADOW - you won't see it in the window styles because it's a class style, not a window style.