I want to evaluate a software solution where multiple people have submitted JAR files to perform a task on Windows.
Is it possible to check whether the JAR file performs any additional unwanted behaviors besides those it claims to perform on your machine?
First, you can use a JVM set with SecurityManager to do run your application in a way that it can have limited access to sensitive functions.
You can also set up a "sandbox" so the jar cannot have permissions outside of the sandbox... you could use chroot or a similar tool in a linux/unix environment.