Looking for a working example of addTimedTextSource for adding subtitle to a video from an .srt file in Android 4.1

Question

I have been trying to use an .srt file for a timed text source (Only available in android 4.1+ http://developer.android.com/about/versions/android-4.1.html#Multimedia) . The first problem has to do with getting a file descriptor for the .srt file (in the assets folder, how else would you bundle it in your app?). The file gets compressed automatically so you won't be able to even see the file without changing compile settings or doing a custom build. The easiest solution was to rename the .srt file to .jpg so that it does not get compressed and the openFD method still works. I am now adding the TimedTextSource with:

_myMP.addTimedTextSource(getAssets().openFd("captions.jpg").getFileDescriptor(),   MediaPlayer.MEDIA_MIMETYPE_TEXT_SUBRIP);

Now the file loads correctly and using myMP.getTrackInfo() to get a list of tracks, can see that after adding the timed text source, the 6th track has type "3" which is timed text track type. I have used selectTrack to choose this track as said in the google documentation but after doing so no captions ever appear and on my TimedTextListener:

 _myMP.setOnTimedTextListener(new OnTimedTextListener(){
        @Override
        public void onTimedText(MediaPlayer mp, TimedText text) {
                if (text!=null)
                   Log.d("TimedText", text.getText());  
            }       
        });

Fires only once (I have like 20 timed text events in the file) but the text parameter is always null. I have done searches and cannot find a single working code example of using timeText and it does not appear in any sample projects, there is literally no documentation other than the api docs from google but as far as I can tell, NO one has posted a working example of it yet. I am testing this on a google Nexus updated to Android 4.2

Answer 1

I was able to get this to work and since it is still an open question I will include the complete solution here.

Although the idea of changing the file extension to prevent the compression is nice, but I prefer to copy the srt file from the resources to the app local directory on the device, but anyways for the sake of completeness here is a list of extensions that won't be compressed.

".jpg", ".jpeg", ".png", ".gif", ".wav", ".mp2", ".mp3", ".ogg", ".aac", ".mpg", ".mpeg", ".mid", ".midi", ".smf", ".jet", ".rtttl", ".imy", ".xmf", ".mp4", ".m4a", ".m4v", ".3gp", ".3gpp", ".3g2", ".3gpp2",".amr", ".awb", ".wma", ".wmv"

The solution steps are simple:

1. Create a MediaPlayer instance and prepare it by either calling MediaPlayer.create() or player.setDataSource() then player.prepare()

2. If the subtitle files does not already exists on the android device, copy it from the resource folder to the device

3. Call player.addTimedTextSource() with the first argument a String that contains the full path of the subtitle file on the device and MediaPlayer.MEDIA_MIMETYPE_TEXT_SUBRIP as the second argument

4. Select the TimedText track by calling player.selectTrack() and pass the index of timedTextType by searching the TrackInfo[] returned from player.getTrackInfo() (I find it usually 2)

5. Set up a listener with player.setOnTimedTextListener() and then start playing the media file player.start()

Here is the complete class:

To run this exact class you will need two files under your res/raw folder sub.srt and video.mp4 (or whatever extensions). Then define a TextView with the id txtDisplay. Finally your project/device/emulator must support API 16

public class MainActivity extends Activity implements OnTimedTextListener {
    private static final String TAG = "TimedTextTest";
    private TextView txtDisplay;
    private static Handler handler = new Handler();

    @Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    txtDisplay = (TextView) findViewById(R.id.txtDisplay);
    MediaPlayer player = MediaPlayer.create(this, R.raw.video);
    try {
        player.addTimedTextSource(getSubtitleFile(R.raw.sub),
                MediaPlayer.MEDIA_MIMETYPE_TEXT_SUBRIP);
        int textTrackIndex = findTrackIndexFor(
                TrackInfo.MEDIA_TRACK_TYPE_TIMEDTEXT, player.getTrackInfo());
        if (textTrackIndex >= 0) {
            player.selectTrack(textTrackIndex);
        } else {
            Log.w(TAG, "Cannot find text track!");
        }
        player.setOnTimedTextListener(this);
        player.start();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

private int findTrackIndexFor(int mediaTrackType, TrackInfo[] trackInfo) {
    int index = -1;
    for (int i = 0; i < trackInfo.length; i++) {
        if (trackInfo[i].getTrackType() == mediaTrackType) {
            return i;
        }
    }
    return index;
}

private String getSubtitleFile(int resId) {
    String fileName = getResources().getResourceEntryName(resId);
    File subtitleFile = getFileStreamPath(fileName);
    if (subtitleFile.exists()) {
        Log.d(TAG, "Subtitle already exists");
        return subtitleFile.getAbsolutePath();
    }
    Log.d(TAG, "Subtitle does not exists, copy it from res/raw");

    // Copy the file from the res/raw folder to your app folder on the
    // device
    InputStream inputStream = null;
    OutputStream outputStream = null;
    try {
        inputStream = getResources().openRawResource(resId);
        outputStream = new FileOutputStream(subtitleFile, false);
        copyFile(inputStream, outputStream);
        return subtitleFile.getAbsolutePath();
    } catch (Exception e) {
        e.printStackTrace();
    } finally {
        closeStreams(inputStream, outputStream);
    }
    return "";
}

private void copyFile(InputStream inputStream, OutputStream outputStream)
        throws IOException {
    final int BUFFER_SIZE = 1024;
    byte[] buffer = new byte[BUFFER_SIZE];
    int length = -1;
    while ((length = inputStream.read(buffer)) != -1) {
        outputStream.write(buffer, 0, length);
    }
}

// A handy method I use to close all the streams
private void closeStreams(Closeable... closeables) {
    if (closeables != null) {
        for (Closeable stream : closeables) {
            if (stream != null) {
                try {
                    stream.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

@Override
public void onTimedText(final MediaPlayer mp, final TimedText text) {
    if (text != null) {
        handler.post(new Runnable() {
            @Override
            public void run() {
                int seconds = mp.getCurrentPosition() / 1000;

                txtDisplay.setText("[" + secondsToDuration(seconds) + "] "
                        + text.getText());
            }
        });
    }
}

// To display the seconds in the duration format 00:00:00
public String secondsToDuration(int seconds) {
    return String.format("%02d:%02d:%02d", seconds / 3600,
            (seconds % 3600) / 60, (seconds % 60), Locale.US);
}
}

And here is the subtitle file I am using as example:

1
00:00:00,220 --> 00:00:01,215
First Text Example

2
00:00:03,148 --> 00:00:05,053
Second Text Example

3
00:00:08,004 --> 00:00:09,884
Third Text Example

4
00:00:11,300 --> 00:00:12,900
Fourth Text Example

5
00:00:15,500 --> 00:00:16,700
Fifth Text Example

6
00:00:18,434 --> 00:00:20,434
Sixth Text Example

7
00:00:22,600 --> 00:00:23,700
Last Text Example

Here are few screenshots from the test app showing that the TextView is changing automatically (i.e. reading from the subtitle file) as the media file progresses

Edit:

Here is the code for an example project