Is it possible to run average java app server on e.g. jboss or glassfish server on Raspberry pi? Any limitations? Does anyone have an experience in Java development for raspberry pi?
At least any appropriate JVM for RPi will be nice.
It's absolutely possible, but should work better with newer versions having more memory (e.g. the 512 MB version of the Pi). In December 2012 Oracle released a Developer Preview of Java SE 8 for ARM. The blog post with the announcement contains links to a number of tutorials which should get you started with plain Java SE or JavaFX, depending on how fancy your interfaces should be.
If you are concerned about the performance of JavaFX on the Raspberry Pi, check the blog post "JavaFX on Raspberry PI", which contains some information about the performance of JavaFX, comparing it to native applications.
Update: Test results after running Tomcat 7 with Java 8 on the Pi
After I got the 512mb version of the Pi (MODELL B), I installed Java 8 and Tomcat 7.0 without any problems. Tomcat examples were working, and I was able to use the Tomcat manager application without any problems.
With 64 MB of RAM used for the graphics card, I still had 291 MB of free memory (with sshd running):
pi@raspberrypi ~/ $ free -m
total used free shared buffers cached
Mem: 438 146 291 0 10 74
-/+ buffers/cache: 60 377
Swap: 99 0 99
pmap
reported a total of 192 MB being used by the Tomcat process.