Scene change/shot detection/image extraction using ffmpeg from video

Question

I am trying to get representative frames for a video, so as to remove redundant frames which might appear in a video. This is what I use to get the frames.

./ffmpeg -i video.mp4 -vf select="eq(pict_type\,PICT_TYPE_I)" -vsync 2 -s 320x240 thumb-%02d.png

I have also tried

./ffmpeg -i video.mp4 -f image2 -vf "select=gt(scene\,.4)" -vsync vfr thumb%04d.png

The major issue in this is blur. If I just sample frames every 5 seconds, I don't see any blur, however using the above two commands I get a lot of blur.

The video can be found here, http://www.cs.umd.edu/~bharat/video.mp4

In order to sample the video every 10 seconds, I use the following:

./ffmpeg -i video.mp4 -r 1/10 filename%03d.jpg

Output using normal sampling:

Output using select:

However, normal sampling may be bad for some videos, and may create redundant frames. Is there a way I could use some option in ffmpeg and get frames without this blur? If normal sampling can get good frames, there should be frames in the vicinity without blur. I have looked at options such as scenecut in ffmpeg however I am not familiar with using them for this application.

Answer 1

I suggest also taking a look at other freely available shot detection implementations. For example, a custom threshold of 47 with Johan Mathe's Shotdetect yields the following results (first frame of every shot):

However, your question seems to deal more with the problem of video stabilization than it does scene change or shot detection. The blur that you see above as well as in the OP are "artifacts" inherent to the video and are not due to the algorithms used to cut or sample the video. If you want to reduce the amount of motion you should look into this post as well as look at OpenCV's Video Stabilization module. There is also a lot of research addressing this challenge including this and this.