Capturing video out of an OpenGL window in Windows
I am supposed to provide my users a really simple way of capturing video clips out of my OpenGL application's main window. I am thinking of adding buttons and/or keyboard shortcuts for starting and stopping the capture; when starting, I could ask for a filename and other options, if any. It has to run in Windows (XP/Vista), but I also wouldn't like to close the Linux door which I've so far been able to keep open.
The application uses OpenGL fragment and shader programs, the effects due to which I absolutely need to have in the eventual videos.
It looks to me like there might be even several different approaches that could potentially fulfill my requirements (but I don't really know where I should start):
An encoding library with functions like startRecording(filename), stopRecording, and captureFrame. I could call captureFrame() after every frame rendered (or every second/third/whatever). If doing so makes my program run slower, it's not really a problem.
A standalone external program that can be programmatically controlled from my application. After all, a standalone program that can not be controlled almost does what I need... But as said, it should be really simple for the users to operate, and I would appreciate seamlessness as well; my application typically runs full-screen. Additionally, it should be possible to distribute as part of the installation package for my application, which I currently prepare using NSIS.
Use the Windows API to capture screenshots frame-by-frame, then employ (for example) one of the libraries mentioned here. It seems to be easy enough to find examples of how to capture screenshots in Windows; however, I would love a solution which doesn't really force me to get my hands super-dirty on the WinAPI level.
Use OpenGL to render into an offscreen target, then use a library to produce the video. I don't know if this is even possible, and I'm afraid it might not be the path of least pain anyway. In particular, I would not like the actual rendering to take a different execution path depending on whether video is being captured or not. Additionally, I would avoid anything that might decrease the frame rate in the normal, non-capture mode.
If the solution were free in either sense of the word, then that would be great, but it's not really an absolute requirement. In general, the less bloat there is, the better. On the other hand, for reasons beyond this question, I cannot link in any GPL-only code, unfortunately.
Regarding the file format, I cannot expect my users to start googling for any codecs, but as long as also displaying the videos is easy enough for a basic-level Windows user, I don't really care what the format is. However, it would be great if it were possible to control the compression quality of the output.
Just to clarify: I don't need to capture video from an external device like camcorder, nor am I really interested in mouse movements, even though getting them does not harm either. There are no requirements regarding audio; the application makes no noise whatsoever.
I write C++ using Visual Studio 2008, for this very application also taking benefit of GLUT and GLUI. I have a solid understanding regarding C++ and linking in libraries and that sort of stuff, but on the other hand OpenGL is quite new for me: so far, I've really only learnt the necessary bits to actually get my job done.
I don't need a solution super-urgently, so feel free to take your time :)
Answer
There are two different questions here - how to grab frames from an OpenGL application, and how to turn them into a movie file.
The first question is easy enough; you just grab each frame with glReadPixels() (via a PBO if you need the performance).
The second question is a little harder since the cross-platform solutions (ffmpeg) tend to be GPL'd or LGPL'd. Is LGPL acceptable for your project? The Windows way of doing this (DirectShow) is a bit of a headache to use.
Edit: Since LGPL is ok and you can use ffmpeg, see here for an example of how to encode video.