When I run my tests in XCode 5, the main window of my OS X app appears on the screen for a couple of seconds while running the tests. Why? Even if I uncomment all my tests it still opens my main window.
You are running application test, not logic test. This means an instance of your app will be started and then run the unit tests. This allow you to perform some integration test that require your app is running.
Here is the guide to setup application test and logic test.
If you want to change it to logic test (so it run faster and don't need to start your app first):
Bundle