jps returns no output even when java processes are running
I'm trying to debug some issues with java processes on a Solaris box, but running jps returns no output. And jstack gives the error 'Permission denied'. The box is part of a cluster of 3 identical servers, jps and jstack work fine on the 2 other servers.
I found the following forum post from someone with the same problem but no answers: http://forums.sun.com/thread.jspa?threadID=5422237
To clarify running bps and grep for java gives all java proceses correctly, but jps gives nothing (anonymised with 'program' and 'client' to protect the guilty):
program @ clientdelivery2 : ~/
-> bps auxww|grep java
program 3427 5.5 54.067742726649544 ? S Sep 25 1039:47 /usr/jdk/instances/jdk1.6.0_16/bin/amd64/java -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=/app/client/program/tomcat/conf/logging.properties -Xmx6144m -XX:PermSize=128m -XX:MaxPermSize=512m -Djava.endorsed.dirs=/app/client/program/tomcat/endorsed -classpath :/app/client/program/tomcat/bin/bootstrap.jar -Dcatalina.base=/app/client/program/tomcat -Dcatalina.home=/app/client/program/tomcat -Djava.io.tmpdir=/app/client/program/tomcat/temp org.apache.catalina.startup.Bootstrap start
program 29915 0.1 11.915252441467896 ? S 14:55:28 3:59 /usr/jdk/instances/jdk1.6.0_16/bin/amd64/java -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=/app/clientclone/program/tomcat/conf/logging.properties -Xmx2g -XX:PermSize=128m -XX:MaxPermSize=512m -Dcom.sun.management.jmxremote -Djava.endorsed.dirs=/app/clientclone/program/tomcat/endorsed -classpath :/app/clientclone/program/tomcat/bin/bootstrap.jar -Dcatalina.base=/app/clientclone/program/tomcat -Dcatalina.home=/app/clientclone/program/tomcat -Djava.io.tmpdir=/app/clientclone/program/tomcat/temp org.apache.catalina.startup.Bootstrap start
program 1573 0.0 0.0 4760 1332 pts/5 S 17:05:24 0:00 grep --colour java
program @ clientdelivery2 : ~/
-> jps
program @ clientdelivery2 : ~/
->
I asked around and from here http://forums.oracle.com/forums/message.jspa?messageID=5408592 I've got that the issue is:
12460/2: mkdir("/tmp/hsperfdata_program", 0755) Err#13 EACCES [ALL]
Meaning jps is being denied access to the psperfdata directory.
Has anyone run into this issue and know how to solve it?
Answer
Turns out the user didn't have access to /tmp because of some issue with mounting the file system. This leads to the files inside hsperfdata_ to never be written, even though the user had access to the /tmp/hsperfdata_ folder itself.