Why can I stream h264 encoded video from webcam to BOTH display and file, but NOT raw video?

Question

I want to stream raw video from a Logitech C920 webcam and while both displaying and saving the video to file using GStreamer 1.0.

This works if I stream h264 encoded video from the camera (the camera provides hardware encoded h264), but it fails if I stream raw video from the camera. However, if I only display, or only save to file, streaming raw video works.

Why does it work with a h264 video stream but not with a raw video stream?

h264 encoded video stream from camera to BOTH display and file (WORKS):

gst-launch-1.0 -v v4l2src device=/dev/video0 \
    ! video/x-h264,width=640,height=480,framerate=15/1 ! tee name=t \
    t. ! queue ! h264parse ! avdec_h264 ! xvimagesink sync=false \
    t. ! queue ! h264parse ! matroskamux \
    ! filesink location='h264_dual.mkv' sync=false

raw video stream from camera to ONLY display (WORKS):

gst-launch-1.0 -v v4l2src device=/dev/video0 \
    ! video/x-raw,format=YUY2,width=640,height=480,framerate=15/1 \
    ! xvimagesink sync=false

raw video stream from camera to ONLY file (WORKS):

gst-launch-1.0 -v v4l2src device=/dev/video0 \
    ! video/x-raw,format=YUY2,width=640,height=480,framerate=15/1 \
    ! videoconvert ! x264enc ! matroskamux \
    ! filesink location='raw_single.mkv' sync=false

raw video stream from camera to BOTH display and file (FAILS):

gst-launch-1.0 -v v4l2src device=/dev/video0 \
    ! video/x-raw,format=YUY2,width=640,height=480,framerate=15/1 \
    ! tee name=t \
    t. ! queue ! xvimagesink sync=false \
    t. ! queue ! videoconvert ! x264enc ! h264parse ! matroskamux \
    ! filesink location='raw_dual.mkv' sync=false

The last command (raw video to both display and file) fails without any warnings or errors. The gst-launch terminal output is exactly the same as when only writing to file. The xvimage window appears and displays an image from the camera, but the image does not change (i.e. it is frozen). A zero bytes file appears too.

I have tried multiple versions of the above commands, but I think those are the minimal commands that can reproduce the problem.

Does anyone understand what I am doing wrong?

Answer 1

Streaming raw video from a webcam (not specific to C920) to both display and h.264 encoded file can be done. The x264enc property tune needs to be set to zerolatency.

h.264 example:

gst-launch-1.0 -v v4l2src device=/dev/video0 \ ! video/x-raw,format=YUY2,width=640,height=480,framerate=15/1 \ ! tee name=t t. ! queue ! xvimagesink sync=false t. ! queue ! \ videoconvert ! x264enc tune=zerolatency ! h264parse ! \ matroskamux ! filesink location='raw_dual.mkv' sync=false

Alternatively, one can skip h.264 altogether and encode to theora or vp8 instead.

theora example:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! \ video/x-raw,format=YUY2,width=640,height=480,framerate=15/1 ! \ tee name=t t. ! queue ! xvimagesink sync=false t. ! queue ! \ videoconvert ! theoraenc ! theoraparse ! \ matroskamux ! filesink location='raw_dual.mkv' sync=false

vp8 example:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! \ video/x-raw,format=YUY2,width=640,height=480,framerate=15/1 ! \ tee name=t t. ! queue ! xvimagesink sync=false t. ! queue ! \ videoconvert ! vp8enc ! \ matroskamux ! filesink location='raw_dual.mkv' sync=false

Thanks a lot to Jan Spurny and Tim.