What is the best way to debug OpenGL?
I find that a lot of the time, OpenGL will show you it failed by not drawing anything. I'm trying to find ways to debug OpenGL programs, by inspecting the transformation matrix stack and so on. What is the best way to debug OpenGL? If the code looks and feels like the vertices are in the right place, how can you be sure they are?
Answer
There is no straight answer. It all depends on what you are trying to understand. Since OpenGL is a state machine, sometimes it does not do what you expect as the required state is not set or things like that.
In general, use tools like glTrace / glIntercept (to look at the OpenGL call trace), gDebugger (to visualize textures, shaders, OGL state etc.) and paper/pencil :). Sometimes it helps to understand how you have setup the camera and where it is looking, what is being clipped etc. I have personally relied more to the last than the previous two approaches. But when I can argue that the depth is wrong then it helps to look at the trace. gDebugger is also the only tool that can be used effectively for profiling and optimization of your OpenGL app.
Apart from this tool, most of the time it is the math that people get wrong and it can't be understood using any tool. Post on the OpenGL.org newsgroup for code specific comments, you will be never disappointed.