How to reduce latency when streaming x264

Question

I would like to produce a zerolatency live video stream and play it in VLC player with as little latency as possible.

This are the settings I currently use:

x264_param_default_preset( &m_Params, "veryfast", "zerolatency" );

m_Params.i_threads              =   2;
m_Params.b_sliced_threads       =   true;
m_Params.i_width                =   m_SourceWidth;
m_Params.i_height               =   m_SourceHeight;

m_Params.b_intra_refresh        =   1;

m_Params.b_vfr_input            =   true;
m_Params.i_timebase_num         =   1;
m_Params.i_timebase_den         =   1000;

m_Params.i_fps_num              =   1;
m_Params.i_fps_den              =   60;

m_Params.rc.i_vbv_max_bitrate   =   512;
m_Params.rc.i_vbv_buffer_size   =   256;
m_Params.rc.f_vbv_buffer_init   =   1.1f;

m_Params.rc.i_rc_method         =   X264_RC_CRF;
m_Params.rc.f_rf_constant       =   24;
m_Params.rc.f_rf_constant_max   =   35;

m_Params.b_annexb               =   0;
m_Params.b_repeat_headers       =   0;
m_Params.b_aud                  =   0;

x264_param_apply_profile( &m_Params, "high" );

Using those settings, I have the following issues:

• VLC shows lots of missing frames (see screenshot, "verloren"). I am not sure if this is an issue.
• If I set a value <200ms for the network stream delay in VLC, VLC renders a few frames and than stops to decode/render frames.
• If I set a value >= 200ms for the network stream delay in VLC, everything looks good so far but the latency is, obviously, 200ms, which is too high.

Question: Which settings (x264lib and VLC) should I use in order to encode and stream with as little latency as possible?

Answer 1

On your x264 settings: many are redundant ie already contained in "zerolatency". However, as best as I can tell, your encoding latency is nevertheless zero frames, ie you put one frame in and you immediately (as soon as your CPU has finished encoding it, anyway) get one frame out. It never waits for a newer frame in order to give an encoded older frame (the way it would with lookahead, for example).

On why vlc pauses unless you give it a large network delay: The problem is that your combination of rate control and vbv settings when encoding is not ideal. What you want to do for low latency encode is to use CBR, and set the VBV buffer to the size of one frame, exactly. This enables a special VBV calculation, if you look in the x264 source.

You may also try not setting anything timing related whatsoever (no fps, no vbv) and use CRF with zerolatency. The results would depend on what container the video is packaged in for streaming.

Read this for more info: http://x264dev.multimedia.cx/archives/249