The Composition Time(CTS) when wrapping H.264 NALU's
The h.264 hardware compression card procedures NALU's from captured video.
I am trying to wrap the NALU's into FLV and I almost succeed.
I don't know how to fill the Composition Time field in FLV for each NALU.
According to the FLV spec, http://download.macromedia.com/f4v/video_file_format_spec_v10_1.pdf, E.4.3.1.
CompositionTime Composition time offset
See ISO 14496-12, 8.15.3 for an explanation of composition times. The offset in an FLV file is always in milliseconds
Then look into the ISO 14496-12,8.15.3 , Page 24 and 26
provides the offset between decoding time and composition time. Since decoding time must be less than the composition time, the offsets are expressed as unsigned numbers such that CT(n) = DT(n) + CTTS(n) where CTTS(n) is the (uncompressed) table entry for sample n.
How can I know the DT and CTTS in each NALU? or how to caculate the CT without DT and CTTS?
Thank you
Answer
For MPEG-4 H.264 transcoders that deliver I-frame, P-frame, and B-frame NALUs inside an MPEG-2 transport, the resulting packetized elementary streams (PES) are timestamped with presentation time stamps (PTS) and decoder timestamps (DTS) in time units of 1/90000 of a second.
The NALUs come in DTS timestamp order in a repeating pattern like
I P B B B P B B B ...
where the intended playback rendering is
I B B B P B B B P ...
(This transport strategy ensures that both frames that the B-frame bridges are in the decoder before the B-frame is processed.)
For FLV, the Timestamp (FLV spec p.69) tells when the frame should be fed to the decoder in milliseconds, which is
timestamp = DTS / 90.0
The CompositionTime (FLV spec p.72) tells the renderer when to perform ("compose") the video frame on the display device in milliseconds after it enters the decoder; thus it is
compositionTime = (PTS - DTS) / 90.0
(Because the PTS >= DTS, this delta is never negative.)