Android Speech Speech Recognition: Repeated Calling of SpeechRecognizer.startListening() fails on JB 4.1.2
I have a service where I'm kicking off a speech recognition listener repeatedly so I can have an open ended session for the user to speak. The class also handles the issue in Jelly Bean where a ERROR_SPEECH_TIMEOUT is thrown if no speech is heard in 5 seconds. So basically this works. However if I repeatedly call recognizer.startListening(recognizerIntent) , it silently fails as evidenced by the fact that onBeginningOfSpeech() is never called in this case. Now IF I just do not talk at all my Jelly Bean timeout handler will restart the listener every time without fail. It seems to only fail after onResults() is called because speech WAS heard. After onResults() is called, recognizer.startListening(recognizerIntent) is definitely called, but like I said, nothing happens. When it fails it is random and there is nothing in Logcat to indicate what the issue is. I just don't know what else to try. Hopefully one of you Android Speech recognition experts out there has seen this before...
Minimum: 2.2 Target: Testing on JB : Android 4.1.2
MORE INFO (11-01-13) The 4.3 Update to my HTC One has definitely addressed this issue. My Speech recognition service below is now reliable & accurate. I has run for at least few minutes without error. As for 4.1.2, before my update to 4.3 it seemed to have been working better (did Google change anything on their end?)...I don't know, but it still would miss some words spoken and occasionally without error, just stop listening (onBeginningOfSpeech() is never called in this case). I suppose I will just have to warn my users about Android 4.1.2 with regards to this issue because I have gone as far as I can go in my code.
MORE INFO (09-17-13) Supposedly, there is an Android update (4.3) coming to HTC ones at the end of September (http://www.ubergizmo.com/2013/09/htc-one-to-receive-android-4-3-jelly-bean-update-this-september/). So hopefully that will address this issue on that device. The issue remains though for my app users that are running Android 4.1.2 and stuck on that version for a while. I still don't know what to do in those cases, and hopefully that is the ONLY Android version with this issue. Is there any way to find out how many devices are running 4.1.2??
MORE INFO (09-15-13) In this post here: Google voice recognizer doesn't starts on Android 4.x. The author state states that he is seeing this issue on his HTC one. I also have an HTC one that I'm seeing this issue on (Android 4.1.2) . I wonder if this is unique to the HTC one? (or any device running Android 4.1.2) - I can't confirm as its difficult to test on all the latest devices running JB. The author further states that his Nexxus with 4.2.2 works fine. Can anyone tell me on what device they see this issue?
MORE INFO (9-08-13) Just to confirm there is no issue with my code, I also tested this on Android 2.3.3 and I was able to call onResult() > startListening() 25 times in a row. When targeting Android 4.1.2 I'm never able to get past 3 or 4 calls. I can't believe no one else has run into this issue?
public class VoiceRecogService extends Service
{
protected AudioManager mAudioManager;
protected SpeechRecognizer mSpeechRecognizer;
protected Intent mSpeechRecognizerIntent;
protected RecognitionListener mSpeechRecognizerListner;
//protected final Messenger mServerMessenger = new Messenger(new IncomingHandler(this));
protected volatile boolean mIsListening;
protected volatile boolean mIsCountDownOn;
static final int MSG_RECOGNIZER_START_LISTENING = 1;
static final int MSG_RECOGNIZER_CANCEL = 2;
private int mBindFlag;
private Messenger mServiceMessenger;
private Context m_ctx;
private Handler mHandler = new Handler();
//private boolean m_bReadyForSpeechReceived = false;
@Override
public void onCreate()
{
super.onCreate();
m_ctx = this;
mAudioManager = (AudioManager) getSystemService(Context.AUDIO_SERVICE);
//do not mute beep when speech listening first kicks off
Log.d("TESTING: SPEECH SERVICE: CALL START", "onCreate()");
startListening(false);
}
private void startListening(boolean bMuteSound){
Log.d("TESTING: SPEECH SERVICE: startListening()", mIsListening? "true":"false");
if (bMuteSound==true && Build.VERSION.SDK_INT >= 16)//Build.VERSION_CODES.JELLY_BEAN)
{
// turn off beep sound
mAudioManager.setStreamMute(AudioManager.STREAM_SYSTEM, true);
}
if (!mIsListening)
{
//mSpeechRecognizer.startListening(mSpeechRecognizerIntent);
recognizeSpeechDirectly ();
mIsListening = true;
}
}
/////////////////////////////////////////////////////////////////////////
/**
* lazy initialize the speech recognizer
*/
private SpeechRecognizer getSpeechRecognizer()
{
if (mSpeechRecognizer == null)
{
mSpeechRecognizer = SpeechRecognizer.createSpeechRecognizer(m_ctx);
}
return mSpeechRecognizer;
}
private RecognitionListener getSpeechRecognizerListner()
{
if (mSpeechRecognizerListner == null)
{
mSpeechRecognizerListner = new SpeechRecognitionListener();
}
return mSpeechRecognizerListner;
}
private void recognizeSpeechDirectly()
{
Intent recognizerIntent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
recognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
// accept partial results if they come
recognizerIntent.putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, true);
recognizeSpeechDirectly(m_ctx,recognizerIntent, getSpeechRecognizerListner(), getSpeechRecognizer());
}
public static void recognizeSpeechDirectly(Context context,
Intent recognizerIntent,
RecognitionListener listener,
SpeechRecognizer recognizer)
{
//need to have a calling package for it to work
if (!recognizerIntent.hasExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE))
{
recognizerIntent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE, "com.dummy");
}
recognizer.setRecognitionListener(listener);
recognizer.startListening(recognizerIntent);
}
////////////////////////////////////////////////////////////////////////////
public void stop()
{
if (getSpeechRecognizer() != null)
{
getSpeechRecognizer().stopListening();
getSpeechRecognizer().cancel();
getSpeechRecognizer().destroy();
mIsListening = false;
if (Build.VERSION.SDK_INT >= 16);//Build.VERSION_CODES.JELLY_BEAN)
mAudioManager.setStreamMute(AudioManager.STREAM_SYSTEM, false);
}
}
// Count down timer for Jelly Bean work around
protected CountDownTimer mNoSpeechCountDown = new CountDownTimer(5000, 5000)
{
@Override
public void onTick(long millisUntilFinished)
{
// TODO Auto-generated method stub
}
@Override
public void onFinish()
{
mIsCountDownOn = false;
Log.d("TESTING: SPEECH SERVICE: CALL START", "onFinish()");
startListening(true);
}
};
@Override
public void onDestroy()
{
super.onDestroy();
if (mIsCountDownOn)
{
mNoSpeechCountDown.cancel();
}
if (mSpeechRecognizer != null)
{
mSpeechRecognizer.destroy();
}
}
protected class SpeechRecognitionListener implements RecognitionListener
{
@Override
public void onReadyForSpeech(Bundle params)
{
if (Build.VERSION.SDK_INT >= 16)//Build.VERSION_CODES.JELLY_BEAN)
{
mIsCountDownOn = true;
mNoSpeechCountDown.start();
}
Log.d("TESTING: SPEECH SERVICE", "onReadyForSpeech");
}
@Override
public void onBeginningOfSpeech()
{
// speech input will be processed, so there is no need for count down anymore
if (mIsCountDownOn)
{
mIsCountDownOn = false;
mNoSpeechCountDown.cancel();
}
}
@Override
public void onEndOfSpeech()
{
Log.d("TESTING: SPEECH SERVICE", "onEndOfSpeech");
}
@Override
public void onBufferReceived(byte[] buffer)
{
//Log.d("TESTING: SPEECH SERVICE", buffer + new String(new byte[] {0x63}));
}
@Override
public void onError(int error)
{
if ((error == SpeechRecognizer.ERROR_NO_MATCH)
|| (error == SpeechRecognizer.ERROR_SPEECH_TIMEOUT)){
if (mIsCountDownOn)
{
mIsCountDownOn = false;
mNoSpeechCountDown.cancel();
}
mIsListening = false;
Log.d("TESTING: SPEECH SERVICE: CALL START", "onError()");
startListening(true);
}
}
@Override
public void onEvent(int eventType, Bundle params)
{
}
@Override
public void onPartialResults(Bundle partialResults)
{
}
@Override
public void onResults(Bundle results)
{
//String str = new String();
//Log.d(TAG, "onResults " + results);
ArrayList data = results.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
//if(data.size() >=1){
// //check for save it:
//}
for (int i = 0; i < data.size(); i++)
{
Log.d("TESTING: SPEECH SERVICE", (String)data.get(i));
}
//if no "save it" somewhere in there, then continue:
if (mIsCountDownOn)
{
mIsCountDownOn = false;
}
mIsListening = false;
Log.d("TESTING: SPEECH SERVICE: CALL START", "onResults()");
startListening(true);
}
@Override
public void onRmsChanged(float rmsdB)
{
}
}
@Override
public IBinder onBind(Intent arg0) {
// TODO Auto-generated method stub
return null;
}
}
Answer
I am on Android 4.4 KitKat on a Nexus 5 and it has this same issue. I think it's likely an Android bug because I haven't seen anyone with a clean solution.
This solution is similar to Andrew_CS solution but I think it's actually allows for better recognition. Andrews solution is constantly starting and stopping the recognizer and you have to keep track of the state of things like if you are processing speech or not. This new solution/work-around basically does this:
- Once onResults is called and our results are processed, we start a timer.
- If things are working properly, onReadyForSpeech will be called and we can cancel our timer.
- If things are not working properly, our timer finishes and we restart the speech recognizer and start the timer again.
- Make sure you cancel the timer in your OnDestroy method as well.
Please let me know if you find an even better way, but this seems to work really well for now. If we can actually prove this is an Android bug, I'd love for someone to submit it to Google.
@Override
public void onReadyForSpeech(Bundle params) {
Log.d("Speech", "onReadyForSpeech: Cancel Timer");
if(mTimer != null) {
mTimer.cancel();
}
}
@Override
public void onResults(Bundle results) {
//If the timer is available, cancel it so it doesn't interrupt our result processing
if(mTimer != null){
mTimer.cancel();
}
Log.d("Speech", "onResults");
//Start processing data
ArrayList<String> strlist = results.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
for (int i = 0; i < strlist.size();i++ ) {
Log.d("Speech", "YOU SAID: " + strlist.get(i));
}
//Start listening again
Log.d("Speech", "onResults: Start Listening");
mSpeechRecognizer.startListening(mRecognizerIntent);
//Start a timer in case OnReadyForSpeech is never called back (Android Bug?)
Log.d("Speech", "onResults: Start a timer");
if(mTimer == null) {
mTimer = new CountDownTimer(2000, 500) {
@Override
public void onTick(long l) {
}
@Override
public void onFinish() {
Log.d("Speech", "Timer.onFinish: Timer Finished, Restart recognizer");
mSpeechRecognizer.cancel();
mSpeechRecognizer.startListening(mRecognizerIntent);
}
};
}
mTimer.start();
}