How can i mux H264 stream into MP4 file via libavformat

Question

I want to realize an application that firstly decode a multi-media file(such as test.mp4 file, video codec id is H264), get a video stream and an audio stream, then make some different in the audio stream, at last encode the video stream(use libx264) and audio stream into a result file(result.mp4). To promote the efficiency, i omitted the decode and encode of video stream, i get the video packet via function "av_read_frame", then output it directly into the result file via function "av_write_frame". But there is no picture in the output file, and the size of output file is fairly small.

I tracked the ffmpeg code and found that in the function "av_write_frame->mov_write_packet->ff_mov_write_packet", it will call function "ff_avc_parse_nal_units" to obtain the size of nal unit, but the return value is very small(such as 208 bytes).

I find that the H264 stream in the MP4 file is not stored in Annex-B format, so it can't find start code(0x000001）， now my problem is how can I change the H264 stream to Annex-B format, and make it work? I added start code at the beginning of every frame manually, but it still not work.

Anyone can give me any hint?Thanks very much. Following is the codes similar with my:

    // write the stream header, if any
    av_write_header(pFormatCtxEnc);

    .........
    /**
    * Init of Encoder and Decoder
    */

    bool KeyFlag = false;
    bool KeyFlagEx = false;
    // Read frames and save frames to disk
    int iPts = 1;
    av_init_packet(&packet);
    while(av_read_frame(pFormatCtxDec, &packet)>=0)
    {
            if (packet.flags == 1)
                    KeyFlag = true;

            if (!KeyFlag)
                    continue;

            if (m_bStop)
            {
                    break;
            }

            // Is this a packet from the video stream?
            if(packet.stream_index == videoStream)
            {
                    currentframeNum ++;
                    if (progressCB != NULL && currentframeNum%20 == 0)
                    {
                            float fpercent = (float)currentframeNum/frameNum;
                            progressCB(fpercent,m_pUser);
                    }

                    if (currentframeNum >= beginFrame && currentframeNum <= endFrane)
                    {
                            if (packet.flags == 1)
                                    KeyFlagEx = true;

                            if (!KeyFlagEx)
                                    continue;
                            packet.dts = iPts ++;
                            av_write_frame(pFormatCtxEnc, &packet);

                    }
            }
            // Free the packet that was allocated by av_read_frame
    }

    // write the trailer, if any
    av_write_trailer(pFormatCtxEnc);

    /**
    * Release of encoder and decoder
    */

    return true;

Answer 1

You might try this: libavcodec/h264_mp4toannexb_bsf.c. It converts bitstream without start codes to bitstream with start codes.

Using your source file, does ffmpeg -i src.mp4 -vcodec copy -an dst.mp4 work? Does it work if you add -bsf h264_mp4toannexb? (all using the same version/build of ffmpeg as you are trying to use programmatically of course)