What Java utilities exist for benchmarking a machine's CPU, memory, disk and network I/O performance?
I need to explain why my application is performing poorly at a remote location. It is going to be inconvenient for me to install a profiler. I suspect that the machine is configured poorly but I need some simple way to show that a standard app suffers on their machine, thus absolving my application and shifting the focus to their system administrators.
Ideally, I would like to suggest they install a simple Java app that tests the performance of the host environment. If it could return some spec for CPU, disk, network, etc, they would be perfect.
I expected to find such a thing on the web, but I've been unsuccessful so far in my search.
Answer
As far as a "simple Java app", I previously did something similar. My efforts were focused on the profiling of a Java VM (virtual machine - the "JVM"), not testing the performance of a specific Java application. The focus was on testing the capabilities of the hosting operating system - exercising:
- CPU
- Memory
- Operating System
- If virtualized, the server VM (not JVM) itself - including both the virtualized and physical components of all of the above.
- Largely ignoring external factors, e.g. disk and network.
The solutions I looked at for this included:
SciMark is a composite Java benchmark measuring the performance of numerical codes occurring in scientific and engineering applications. It consists of five computational kernels: FFT, Gauss-Seidel relaxation, Sparse matrix-multiply, Monte Carlo integration, and dense LU factorization.
These kernels are chosen to provide an indication of how well the underlying JVM/JITs perform on applications utilizing these types of algorithms. The problems sizes are purposely chosen to be small in order to isolate the effects of memory hierarchy and focus on internal JVM/JIT and CPU issues. A larger version of the benchmark (SciMark 2.0 LARGE) addresses performance of the memory subsystem with out-of-cache problem sizes.
It measures several computational kernels and reports a composite score in approximate Mflops (Millions of floating point operations per second).
While somewhat dated (2004), SciMark does still appear to be quite effective.
Each test run takes about 30 seconds to execute.
SPECjvm2008 (Java Virtual Machine Benchmark) is a benchmark suite for measuring the performance of a Java Runtime Environment (JRE), containing several real life applications and benchmarks focusing on core java functionality. The suite focuses on the performance of the JRE executing a single application; it reflects the performance of the hardware processor and memory subsystem, but has low dependence on file I/O and includes no network I/O across machines. The SPECjvm2008 workload mimics a variety of common general purpose application computations. These characteristics reflect the intent that this benchmark will be applicable to measuring basic Java performance on a wide variety of both client and server systems.
SPEC also finds user experience of Java important, and the suite therefore includes startup benchmarks and has a required run category called base, which must be run without any tuning of the JVM to improve the out of the box performance.
SPECjvm2008 Benchmark Highlights
- Leverages real life applications (like derby, sunflow, and javac) and area-focused benchmarks (like xml, serialization, crypto, and scimark).
- Also measures the performance of the operating system and hardware in the context of executing the JRE.
Unlike SciMark, SPECjvm2008 takes much longer to fully execute the suite - taking just over 2 hours.
For testing the network, I'd use something like iperf, available for both Linux/Windows/etc.
For testing disk performance, there is an entire science behind this, and without knowing what operating system you're using, I can't give you a specific answer. However, if you're using Linux, for example, I'd review this.