Early Z-test / depth-test in DirectX 11
As a DirectX noob i'm trying to wrap my head around depth buffers and specifically how pixel shaders are called for obscured pixels.
From what i understand, the rasterizer calls the pixel shader for each pixel that is covering the primitive beeing drawn, then later in the output merger stage, the output merger checks the depth buffer and either discards, writes or blends the pixel in the back buffer.
This seems wasteful though if i'm rendering a simple opaque object in front of a very complex one, so it seems it would be useful to have the rasterizer check the depthmap Before even calling the pixel shaders for the complex object.
Doing research i've found references to early Z test / conservative Z testing and so on, but there also seems to be very little documentation about it. i looked for a way to configure that on the rasterizer state desc object, but i only found anything like that on the OM state desc.
It also seems like this was possible to set with SetRenderState in DX9 (i have no experience with DX9 either though)
From my research it seems like this is something some hardware just does if i render objects front to back, is that correct? How can i even tell? With all the Control DirectX gives you it seems weird that there is no control over this, as it seems to be a good optimization :)
Any info or references on this is appriciated
Answer
As far as depth testing goes, DirectX states that it must appear that depth testing happens after the pixel shader, it doesn't say that it actually has to. In reality, early z has existing for many years on many manufacturers' hardware. There's often an even earlier-than-early form of Z-testing called Hierarchical Z that operates not on individual pixels but on 'tiles' of many pixels at a time to avoid the cost of early-Z.
Early Z is not something you can turn on or off through any particular state you set on the device. The hardware will perform z-testing as early as it can and in such a way that you didn't know it was anywhere other than after the pixel shader.
There are certain things you can do though that can restrict the hardware in its ability to do z-testing as early as it might otherwise have been able to. Alpha testing, use of 'discard' (killing the pixel) and alpha to coverage will all certainly disable early-z writes as the pixel shaders need to be run before the hardware can determine whether to write a depth value or not. If you're using alpha testing / discard and don't need z-writes, then turn them off and you stand the best possible chance of early-z being available.
Modifying/writing 'depth' in the pixel shader is a definite no-no if you want early Z. In this situation the hardware can't even perform an early test as it's not yet aware of what the depth of the pixel is until you've decided it in the pixel shader, it can neither perform an early z-test nor an early z write.
If you need to write depth from the pixel shader but can guarantee you'll only be writing a depth value greater than or equal to the one the rasterizer has produced you can use the rather undocumented SV_DepthGreater output semantic. Since you're promising not to write a depth value less than the interpolated depth the hardware can still try and perform an early-z test but then defer the z write until the end of the pixel shader. (There is an SV_DepthLessEqual equivalent if you happen to be using an inverted z-test/z-buffer).
Since it must appear that z-testing happens at the end of the pipeline, using UAVs in the pixel shader will also disable early-z. Since the render target is now not the only output and since the DirectX specification says it must appear that z-testing happens at the end, it follows that your UAV writes should happen even for pixels that will eventually fail the z-test. For this reason an attribute was added in Shader Model 5 called [earlydepthstencil] which tells DirectX that you're happy for early-z to occur (if possible) and not to run the pixel shader even though you've got UAV writes happening.
In summary, if you're not doing any of the slightly quirky things mentioned above (modifying depth using SV_DEPTH, alpha to coverage, using clip/discard etc) then chances are you're already getting the benefit of early-Z. Always turn off z-writes if you don't want them and stay clear of writing to SV_DEPTH as there's absolutely no way you get early-z with that enabled.