I want to decode and display a raw h264 video byte stream in Android and therefore I'm currently using the MediaCodec/Format
classes. I acquire the frame data via Udp from a server.
But, unfortunately, nothing is displayed at the moment.
Here is what I have so far.
Initializing MediaCodec Class:
codec = MediaCodec.createDecoderByType("video/avc");
MediaFormat format = new MediaFormat();
format.setString(MediaFormat.KEY_MIME, "video/avc");
format.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, 100000);
format.setInteger(MediaFormat.KEY_WIDTH, 800);
format.setInteger(MediaFormat.KEY_HEIGHT, 600);
format.setInteger("max-width", 800);
format.setInteger("max-height", 600);
format.setInteger("push-blank-buffers-on-shutdown", 1);
codec.configure(format, surface, null, 0);
Usage of the Decoder:
int inIndex = codec.dequeueInputBuffer(10000);
if(inIndex >= 0)
{
ByteBuffer inputBuffer = codecInputBuffers[inIndex];
inputBuffer.clear();
inputBuffer.put(frameData);
codec.queueInputBuffer(inIndex, 0, frameSize, 33, 0);
}
int outIndex = codec.dequeueOutputBuffer(null, 10000);
switch(outIndex)
{
case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
codecOutputBuffers = codec.getOutputBuffers();
System.out.println("OB Changed");
break;
case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
System.out.println("OF Changed");
break;
case MediaCodec.INFO_TRY_AGAIN_LATER:
System.out.println("l8r");
break;
default:
ByteBuffer buffer = codecOutputBuffers[outIndex];
codec.releaseOutputBuffer(outIndex, true);
}
The Device I'm testing this code on is a Google Nexus 5. When I run this, the outIndex
equals MediaCodec.INFO_TRY_AGAIN_LATER
the whole time.
I've previously written a Client for Notebooks, which works fine, so I guess the h264 Stream from the Server should be OK.
Thanks for your help
Edit: In case someone is encountering the same problem, the proposed amendment (1) of fadden solved the issue. I corrected the code above. Its working now. Another mistake of my shown example was, that you cannot pass null to .dequeueOutputBuffers(...);. You have to do something like
BufferInfo buffInfo = new MediaCodec.BufferInfo();
int outIndex = codec.dequeueOutputBuffer(buffInfo, 10000);
even if you're not interessted in it. ;)
I see a few problems...
(1) You're attempting to replace a buffer in the input buffer array. MediaCodec
doesn't work like this -- the framework provides the buffers, and you copy the data into them. The idea is that, by allowing the framework to do the allocation, it can potentially avoid copying the data later on.
You need to get the array of input buffers from decoder.getInputBuffers()
, and use those. Make sure to clear()
the ByteBuffer
to reset the position and limit each time.
(2) You're writing a single packet of data and expecting a frame of output data. In practice, you may need to supply multiple buffers of data before the first frame is generated. See this post for an example. In some profiles the encoder is allowed to reorder frames, so even after the decoder starts going you can't just feed a frame and wait for decoded data to pop out the other side.
(3) The AVC decoder needs the SPS/PPS data, which you can provide via a buffer with the BUFFER_FLAG_CODEC_CONFIG
flag set, or by adding the data with "csd-0" / "csd-1" keys to the MediaFormat
using MediaFormat#setByteBuffer()
. Examples of both approaches can be found in EncodeDecodeTest.
There are a number of AVC decoding examples on bigflake, but the data source is the MediaCodec
encoder, so they generally get point #3 for free.
This posting may be useful for you.
For displaying the frames, you can see different approaches in Grafika (which generally works with .mp4 files, so the encode/decode implementation there isn't as relevant).