So I just finished setting up a JBoss application server gear on Openshift and I attached a MySQL and phpmyadmin cartridges. My question is if there is a way to remote access to the database server using an app like MySQL Workbench?
You can use rhc port-forward
to forward ports from your database to your local machine. Check out the tutorial here. The basic idea of port forwarding in this context is that you can forward ports on your local machine to ones on the gear. So in your case, you would forward some port on your local machine to the port mysql is listening on for connections in your gear. Then you would connect MySQL Workbench to the port on your local machine.