What does the HotSpot JVM flag -XX:+UseCompressedOops
do and when should I use it? What sort of performance and memory-usage differences will I see when using it on a 64-bit Java instance (vs. not using it)?
Most HotSpot JVM in the last year have had it on by default. This option allows references to be 32-bit in a 64-bit JVM and access close to 32 GB of heap. (more than 32-bit pointers can) (You can have near unlimited off heap memory as well). This can save a significant amount of memory and potentially improve performance.
If you want to use this option I suggest you update to a version which has it on by default as there may have been a good reason, such as bugs, why it wasn't enabled previously. Try Java 6 update 23 or Java 7 update 5.
In short, don't turn it on, use a version which has it on by default.
Update:
In Java 8 you have the option to set the -XX:ObjectAlignmentInBytes=
and in fact if you heap size to 64 GB it will use -XX:ObjectAlignmentInBytes=16
and still use 32-bit references.