RaspberryPi + Pocketsphinx + ps3eye Error: Failed to open audio device

Question 1

RaspberryPi + Pocketsphinx + ps3eye Error: Failed to open audio device

audio speech-recognition raspberry-pi speech-to-text cmusphinx

Malcolm Jones · Jul 22, 2013 · Viewed 17.7k times · Source

Answer

Answer

Took me a while, with with some help from a couple sources ( they will be listed in my answer ) and some helpful hints from nikolay-shmyrev, I finally came up with an answer that worked for me.

Key assumptions:

running these commands as the pi user ( previously I was running them as root, which was incorrect )
I'm using continuous recognition and I was ONLY looking for the ability to "wake-up" my raspberry pi. Upon waking it up, I have other plans on how it should interact.

My setup:

Canakit RaspberryPi
HDMI cable to my Toshiba TV
usb wifi dongle
Playstation 3 Eye for speech recognition

Moving forward. I ran the following commands on my RaspberryPi to get PulseAudio + pocketsphinx working together w/ my Playstation 3 Eye. ( If you see any places for improvement please let me know )

Install pulse audio / development packages

sudo apt-get install gstreamer0.10-pulseaudio libao4 libasound2-plugins libgconfmm-2.6-1c2 libglademm-2.4-1c2a libpulse-dev libpulse-mainloop-glib0 libpulse-mainloop-glib0-dbg libpulse0 libpulse0-dbg libsox-fmt-pulse paman paprefs pavucontrol pavumeter pulseaudio pulseaudio-dbg pulseaudio-esound-compat pulseaudio-esound-compat-dbg pulseaudio-module-bluetooth pulseaudio-module-gconf pulseaudio-module-jack pulseaudio-module-lirc pulseaudio-module-lirc-dbg pulseaudio-module-x11 pulseaudio-module-zeroconf pulseaudio-module-zeroconf-dbg pulseaudio-utils oss-compat -y

Setting up ALSA

Per instructions from http://forums.debian.net/viewtopic.php?f=16&t=12497

sudo \cp -pf /etc/asound.conf /etc/asound.conf.ORIG 
echo 'pcm.pulse {
    type pulse
}

ctl.pulse {
    type pulse
}

pcm.!default {
    type pulse
}

ctl.!default {
    type pulse
}' | sudo tee /etc/asound.conf

Make sure your camera device loads on boot

_DEVICE_LOAD_ON_START=$(grep "snd.bcm2835" /etc/modules | wc -l)
if [[ "${_DEVICE_LOAD_ON_START}" = "0" ]]; then

  sudo \cp -pf /etc/modules /etc/modules.ORIG
  echo "snd-bcm2835" | tee -a /etc/modules

fi

# Disallow module loading after startup. This is a security feature since it disallows additional module loading during runtime and on user request.
_DISALLOW_MODULE_LOADING=$(grep "DISALLOW_MODULE_LOADING=1" /etc/default/pulseaudio | wc -l)
if [[ "${_DISALLOW_MODULE_LOADING}" = "0" ]]; then

  sudo \cp -pf /etc/default/pulseaudio /etc/default/pulseaudio.ORIG
  sudo sed -i "s,DISALLOW_MODULE_LOADING=1,DISALLOW_MODULE_LOADING=0,g" /etc/default/pulseaudio

fi

Set up the PulseAudio daemon for network connections

# allow other clients on the network to connect to pulseaudio daemon ( only add auth-anonymous=1 if you know EVERY machine on your LAN ... this could be a security risk otherwise )
sudo \cp -fvp /etc/pulse/system.pa /etc/pulse/system.pa.ORIG
echo "
# ScarlettPi ADDED THIS
load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1;192.168.0.0/24 auth-anonymous=1
load-module module-zeroconf-publish" | sudo tee -a /etc/pulse/system.pa

echo "
# ScarlettPi added this
#load-module module-native-protocol-tcp
#load-module module-zeroconf-publish
load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1;192.168.0.0/24 auth-anonymous=1
load-module module-zeroconf-publish" | sudo tee -a /etc/pulse/default.pa

# check to make sure it looks okay
cat /etc/pulse/default.pa

Change default sound driver from alsa to pulseaudio

sudo \cp -fvp /etc/libao.conf /etc/libao.conf.ORIG
sudo sed -i "s,default_driver=alsa,default_driver=pulse,g" /etc/libao.conf 

# daemon settings according to Pi-Musicbox ( https://github.com/woutervanwijk/Pi-MusicBox )
sudo \cp -fvp /etc/pulse/daemon.conf /etc/pulse/daemon.conf.ORIG

echo "
# ScarlettPi added this
high-priority = yes
nice-level = 5
exit-idle-time = -1
resample-method = src-sinc-medium-quality
default-sample-format = s16le
default-sample-rate = 48000
default-sample-channels = 2" | sudo tee -a /etc/pulse/daemon.conf

Add `pi` user to the pulse access group

sudo adduser pi pulse-access

# shut down the machine to make sure all the settings we just made are loaded correctly
sudo shutdown -r now

Make sure to add `/usr/local/lib` to library path

export LD_LIBRARY_PATH=/usr/local/lib
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

# also add these to your .bashrc so they get set once you login 
echo "
# scarlettPi added this
export LD_LIBRARY_PATH=/usr/local/lib
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig" | tee -a ~/.bashrc

Install base PocketSphinx

# install python dev packages
sudo apt-get install python2.7-dev -y

# sphinxbase install ( required to install pocketsphinx )
sudo apt-get install bison -y
cd ~pi/
wget  http://downloads.sourceforge.net/project/cmusphinx/sphinxbase/0.8/sphinxbase-0.8.tar.gz
tar -xvf sphinxbase-0.8.tar.gz
cd sphinxbase-0.8
./configure
make
sudo make install
cd -

# pocketsphinx install
# set this: LD_LIBRARY_PATH=/path/to/pocketsphinxlibs /usr/local/bin/pocketsphinx_continuous
# http://www.voxforge.org/home/forums/message-boards/speech-recognition-engines/howto-use-pocketsphinx
wget http://sourceforge.net/projects/cmusphinx/files/pocketsphinx/0.8/pocketsphinx-0.8.tar.gz
tar -xvf pocketsphinx-0.8.tar.gz
cd pocketsphinx-0.8
./configure
make
sudo make install
cd -

# install sphinxtrain
wget http://sourceforge.net/projects/cmusphinx/files/sphinxtrain/1.0.8/sphinxtrain-1.0.8.tar.gz
tar -xvf sphinxtrain-1.0.8
cd sphinxtrain-1.0.8
./configure
make
sudo make install
cd -

Check if pulse daemon is running

ps aux | grep pulse

# If it isn't, start it up yourself ( need to figure out the best way to make this run on boot...init.d script maybe? )
/usr/bin/pulseaudio --start --log-target=syslog --system=false

Finally, run Sphinx

IMPORTANT NOTE YOU HAVE TO BE USER PI AND THE PULSEAUDIO SERVER NEEDS TO BE RUNNING

Assumimg existing corpus file, .jsgf file, .dic, and .lm files (using lmtool)

cd ~pi/pocketsphinx-0.8
pocketsphinx_continuous -lm /home/pi/scarlettPi/config/speech/lm/scarlett.lm -dict /home/pi/scarlettPi/config/speech/dict/scarlett.dic -hmm /home/pi/scarlettPi/config/speech/model/hmm/en_US/hub4wsj_sc_8k -silprob  0.1 -wip 1e-4 -bestpath 0

References:

I plan on adding more details behind why I used certain setting, configurations in a blog post i'm writing on my home automation project, but figured, i'd share what i've done thus far incase someone else was stuck like me and would like to move forward with what they're working on. Hope this helps someone. Thanks for the advice guys.

Question 2

just installed pocketsphinx on my raspberry pi. Think i'm going crazy but not sure if I'm providing the correct device.

Whenever I run:

src/programs/pocketsphinx_continuous -adcdev plughw:1,0 -nfft 2048 -samprate 48000

I get the following:

root@scarlettpi:/usr/install/pocketsphinx-0.8# src/programs/pocketsphinx_continuous -adcdev plughw:1,0 -nfft 2048 -samprate 48000 INFO: cmd_ln.c(691): Parsing command line: /usr/install/pocketsphinx-0.8/src/programs/.libs/lt-pocketsphinx_continuous \ -adcdev plughw:1,0 \ -nfft 2048 \ -samprate 48000

Current configuration: [NAME] [DEFLT] [VALUE] -adcdev plughw:1,0 -agc none none -agcthresh 2.0 2.000000e+00 -alpha 0.97 9.700000e-01 -argfile
-ascale 20.0 2.000000e+01 -aw 1 1 -backtrace no no -beam 1e-48 1.000000e-48 -bestpath yes yes -bestpathlw 9.5 9.500000e+00 -bghist no no -ceplen 13 13 -cmn current current -cmninit 8.0 8.0 -compallsen no no -debug 0 -dict
-dictcase no no -dither no no -doublebw no no -ds 1 1 -fdict
-feat 1s_c_d_dd 1s_c_d_dd -featparams
-fillprob 1e-8 1.000000e-08 -frate 100 100 -fsg
-fsgusealtpron yes yes -fsgusefiller yes yes -fwdflat yes yes -fwdflatbeam 1e-64 1.000000e-64 -fwdflatefwid 4 4 -fwdflatlw 8.5 8.500000e+00 -fwdflatsfwin 25 25 -fwdflatwbeam 7e-29 7.000000e-29 -fwdtree yes yes -hmm
-infile
-input_endian little little -jsgf
-kdmaxbbi -1 -1 -kdmaxdepth 0 0 -kdtree
-latsize 5000 5000 -lda
-ldadim 0 0 -lextreedump 0 0 -lifter 0 0 -lm
-lmctl
-lmname default default -logbase 1.0001 1.000100e+00 -logfn
-logspec no no -lowerf 133.33334 1.333333e+02 -lpbeam 1e-40 1.000000e-40 -lponlybeam 7e-29 7.000000e-29 -lw 6.5 6.500000e+00 -maxhmmpf -1 -1 -maxnewoov 20 20 -maxwpf -1 -1 -mdef
-mean
-mfclogdir
-min_endfr 0 0 -mixw
-mixwfloor 0.0000001 1.000000e-07 -mllr
-mmap yes yes -ncep 13 13 -nfft 512 2048 -nfilt 40 40 -nwpen 1.0 1.000000e+00 -pbeam 1e-48 1.000000e-48 -pip 1.0 1.000000e+00 -pl_beam 1e-10 1.000000e-10 -pl_pbeam 1e-5 1.000000e-05 -pl_window 0 0 -rawlogdir
-remove_dc no no -round_filters yes yes -samprate 16000 4.800000e+04 -seed -1 -1 -sendump
-senlogdir
-senmgau
-silprob 0.005 5.000000e-03 -smoothspec no no -svspec
-time no no -tmat
-tmatfloor 0.0001 1.000000e-04 -topn 4 4 -topn_beam 0 0 -toprule
-transform legacy legacy -unit_area yes yes -upperf 6855.4976 6.855498e+03 -usewdphones no no -uw 1.0 1.000000e+00 -var
-varfloor 0.0001 1.000000e-04 -varnorm no no -verbose no no -warp_params
-warp_type inverse_linear inverse_linear -wbeam 7e-29 7.000000e-29 -wip 0.65 6.500000e-01 -wlen 0.025625 2.562500e-02

INFO: cmd_ln.c(691): Parsing command line: \ -nfilt 20 \ -lowerf 1 \ -upperf 4000 \ -wlen 0.025 \ -transform dct \ -round_filters no \ -remove_dc yes \ -svspec 0-12/13-25/26-38 \ -feat 1s_c_d_dd \ -agc none \ -cmn current \ -cmninit 56,-3,1 \ -varnorm no

Current configuration: [NAME] [DEFLT] [VALUE] -agc none none -agcthresh 2.0 2.000000e+00 -alpha 0.97 9.700000e-01 -ceplen 13 13 -cmn current current -cmninit 8.0 56,-3,1 -dither no no -doublebw no no -feat 1s_c_d_dd 1s_c_d_dd -frate 100 100 -input_endian little little -lda
-ldadim 0 0 -lifter 0 0 -logspec no no -lowerf 133.33334 1.000000e+00 -ncep 13 13 -nfft 512 2048 -nfilt 40 20 -remove_dc no yes -round_filters yes no -samprate 16000 4.800000e+04 -seed -1 -1 -smoothspec no no -svspec 0-12/13-25/26-38 -transform legacy dct -unit_area yes yes -upperf 6855.4976 4.000000e+03 -varnorm no no -verbose no no -warp_params
-warp_type inverse_linear inverse_linear -wlen 0.025625 2.500000e-02

INFO: acmod.c(246): Parsed model-specific feature parameters from /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/feat.params INFO: feat.c(713): Initializing feature stream to type: '1s_c_d_dd', ceplen=13, CMN='current', VARNORM='no', AGC='none' INFO: cmn.c(142): mean[0]= 12.00, mean[1..12]= 0.0 INFO: acmod.c(167): Using subvector specification 0-12/13-25/26-38 INFO: mdef.c(517): Reading model definition: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/mdef INFO: mdef.c(528): Found byte-order mark BMDF, assuming this is a binary mdef file INFO: bin_mdef.c(336): Reading binary model definition: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/mdef INFO: bin_mdef.c(513): 50 CI-phone, 143047 CD-phone, 3 emitstate/phone, 150 CI-sen, 5150 Sen, 27135 Sen-Seq INFO: tmat.c(205): Reading HMM transition probability matrices: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/transition_matrices INFO: acmod.c(121): Attempting to use SCHMM computation module INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/means INFO: ms_gauden.c(292): 1 codebook, 3 feature, size: INFO: ms_gauden.c(294): 256x13 INFO: ms_gauden.c(294): 256x13 INFO: ms_gauden.c(294): 256x13 INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/variances INFO: ms_gauden.c(292): 1 codebook, 3 feature, size: INFO: ms_gauden.c(294): 256x13 INFO: ms_gauden.c(294): 256x13 INFO: ms_gauden.c(294): 256x13 INFO: ms_gauden.c(354): 0 variance values floored INFO: s2_semi_mgau.c(903): Loading senones from dump file /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/sendump INFO: s2_semi_mgau.c(927): BEGIN FILE FORMAT DESCRIPTION INFO: s2_semi_mgau.c(1022): Using memory-mapped I/O for senones INFO: s2_semi_mgau.c(1296): Maximum top-N: 4 Top-N beams: 0 0 0 INFO: dict.c(317): Allocating 137543 * 20 bytes (2686 KiB) for word entries INFO: dict.c(332): Reading main dictionary: /usr/local/share/pocketsphinx/model/lm/en_US/cmu07a.dic INFO: dict.c(211): Allocated 1010 KiB for strings, 1664 KiB for phones INFO: dict.c(335): 133436 words read INFO: dict.c(341): Reading filler dictionary: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/noisedict INFO: dict.c(211): Allocated 0 KiB for strings, 0 KiB for phones INFO: dict.c(344): 11 words read INFO: dict2pid.c(396): Building PID tables for dictionary INFO: dict2pid.c(404): Allocating 50^3 * 2 bytes (244 KiB) for word-initial triphones INFO: dict2pid.c(131): Allocated 30200 bytes (29 KiB) for word-final triphones INFO: dict2pid.c(195): Allocated 30200 bytes (29 KiB) for single-phone word triphones INFO: ngram_model_arpa.c(77): No \data\ mark in LM file INFO: ngram_model_dmp.c(142): Will use memory-mapped I/O for LM file INFO: ngram_model_dmp.c(196): ngrams 1=5001, 2=436879, 3=418286 INFO: ngram_model_dmp.c(242): 5001 = LM.unigrams(+trailer) read INFO: ngram_model_dmp.c(288): 436879 = LM.bigrams(+trailer) read INFO: ngram_model_dmp.c(314): 418286 = LM.trigrams read INFO: ngram_model_dmp.c(339): 37293 = LM.prob2 entries read INFO: ngram_model_dmp.c(359): 14370 = LM.bo_wt2 entries read INFO: ngram_model_dmp.c(379): 36094 = LM.prob3 entries read INFO: ngram_model_dmp.c(407): 854 = LM.tseg_base entries read INFO: ngram_model_dmp.c(463): 5001 = ascii word strings read INFO: ngram_search_fwdtree.c(99): 788 unique initial diphones INFO: ngram_search_fwdtree.c(147): 0 root, 0 non-root channels, 60 single-phone words INFO: ngram_search_fwdtree.c(186): Creating search tree INFO: ngram_search_fwdtree.c(191): before: 0 root, 0 non-root channels, 60 single-phone words INFO: ngram_search_fwdtree.c(326): after: max nonroot chan increased to 13428 INFO: ngram_search_fwdtree.c(338): after: 457 root, 13300 non-root channels, 26 single-phone words INFO: ngram_search_fwdflat.c(156): fwdflat: min_ef_width = 4, max_sf_win = 25 INFO: continuous.c(371): /usr/install/pocketsphinx-0.8/src/programs/.libs/lt-pocketsphinx_continuous COMPILED ON: Jul 21 2013, AT: 14:34:06

Mixer load failed: Invalid argument FATAL_ERROR: "continuous.c", line 246: Failed to open audio device

I'm using a ps3eye currently. If i do a simple:

arecord -D plughw:1,0 -d 5 -q -f cd -t wav ~/test.wav

Everything works fine ( Verified this by hooking up Raspberrypi to TV via HDMI and running aplay ~/test.wav )

What am I doing wrong guys?

Information you might need ( based on other posts i've seen ):

root@scarlettpi:/usr/install/pocketsphinx-0.8# aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: ALSA [bcm2835 ALSA], device 0: bcm2835 ALSA [bcm2835 ALSA]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
root@scarlettpi:/usr/install/pocketsphinx-0.8# 
root@scarlettpi:/usr/install/pocketsphinx-0.8# aplay -L
null
    Discard all samples (playback) or generate zero samples (capture)
pulse
    PulseAudio Sound Server
sysdefault:CARD=ALSA
    bcm2835 ALSA, bcm2835 ALSA
    Default Audio Device
root@scarlettpi:/usr/install/pocketsphinx-0.8#



root@scarlettpi:/usr/install/pocketsphinx-0.8# dpkg -l | grep "alsa"
ii  alsa-base                             1.0.25+3~deb7u1                      all          ALSA driver configuration files
ii  alsa-firmware-loaders                 1.0.25-2                             armhf        ALSA software loaders for specific hardware
ii  alsa-oss                              1.0.25-1                             armhf        ALSA wrapper for OSS applications
ii  alsa-tools                            1.0.25-2                             armhf        Console based ALSA utilities for specific hardware
ii  alsa-utils                            1.0.25-4                             armhf        Utilities for configuring and using ALSA
ii  alsaplayer-alsa                       0.99.80-5.1                          armhf        PCM player designed for ALSA (ALSA output module)
ii  alsaplayer-common                     0.99.80-5.1                          armhf        PCM player designed for ALSA (common files)
ii  alsaplayer-gtk                        0.99.80-5.1                          armhf        PCM player designed for ALSA (GTK+ version)
ii  gstreamer0.10-alsa:armhf              0.10.36-1.1                          armhf        GStreamer plugin for ALSA
ii  libsox-fmt-alsa                       14.4.0-3                             armhf        SoX alsa format I/O library
root@scarlettpi:/usr/install/pocketsphinx-0.8#



root@scarlettpi:/usr/install/pocketsphinx-0.8# dpkg -l | grep pulseaudio
ii  gstreamer0.10-pulseaudio:armhf        0.10.31-3+nmu1                       armhf        GStreamer plugin for PulseAudio
root@scarlettpi:/usr/install/pocketsphinx-0.8#

Also in terms of installing pocket sphinx I did the following:

# uninstall pulse audio if its already installed 
apt-get remove pulseaudio -y 
aptitude purge pulseaudio -y

# sphinxbase install
apt-get install bison -y
cd /usr/install
wget  http://downloads.sourceforge.net/project/cmusphinx/sphinxbase/0.8/sphinxbase-0.8.tar.gz
tar -xvf sphinxbase-0.8.tar.gz
cd sphinxbase-0.8
./configure
make
make install
cd -

# pocketsphinx installwget http://sourceforge.net/projects/cmusphinx/files/pocketsphinx/0.8/pocketsphinx-0.8.tar.gz
tar -xvf pocketsphinx-0.8.tar.gz
cd pocketsphinx-0.8
./configure
make
make install

Any ideas or advice in the right direction would be extremely helpful.

Thanks,

Malcolm Jones

EDIT:

Forgot to include this information as well:

root@scarlettpi:/usr/install/pocketsphinx-0.8# arecord -L
null
    Discard all samples (playback) or generate zero samples (capture)
pulse
    PulseAudio Sound Server
sysdefault:CARD=CameraB409241
    USB Camera-B4.09.24.1, USB Audio
    Default Audio Device
front:CARD=CameraB409241,DEV=0
    USB Camera-B4.09.24.1, USB Audio
    Front speakers
surround40:CARD=CameraB409241,DEV=0
    USB Camera-B4.09.24.1, USB Audio
    4.0 Surround output to Front and Rear speakers
surround41:CARD=CameraB409241,DEV=0
    USB Camera-B4.09.24.1, USB Audio
    4.1 Surround output to Front, Rear and Subwoofer speakers
surround50:CARD=CameraB409241,DEV=0
    USB Camera-B4.09.24.1, USB Audio
    5.0 Surround output to Front, Center and Rear speakers
surround51:CARD=CameraB409241,DEV=0
    USB Camera-B4.09.24.1, USB Audio
    5.1 Surround output to Front, Center, Rear and Subwoofer speakers
surround71:CARD=CameraB409241,DEV=0
    USB Camera-B4.09.24.1, USB Audio
    7.1 Surround output to Front, Center, Side, Rear and Woofer speakers
iec958:CARD=CameraB409241,DEV=0
    USB Camera-B4.09.24.1, USB Audio
    IEC958 (S/PDIF) Digital Audio Output
root@scarlettpi:/usr/install/pocketsphinx-0.8#

RaspberryPi + Pocketsphinx + ps3eye Error: Failed to open audio device

Answer

Install pulse audio / development packages

Setting up ALSA

Make sure your camera device loads on boot

Set up the PulseAudio daemon for network connections

Change default sound driver from alsa to pulseaudio

Add pi user to the pulse access group

Make sure to add /usr/local/lib to library path

Install base PocketSphinx

Check if pulse daemon is running

Finally, run Sphinx

Related questions

Add `pi` user to the pulse access group

Make sure to add `/usr/local/lib` to library path