I have seen at least three distinct methods on StackOverflow for achieving this.
Using a MUTEX: Accepted answer to this SO question
Using the Microsoft.VisualBasic library's WindowsFormsApplicationBase: Second highest voted answer to this SO question
Using Process.GetProcessNames to check if your application is running: Method here was posted as an answer to this SO question
I'm sure there are more ways to do this as well.
I'm simply wondering if one of these is preferred and what the consequences might be if I pick the "wrong" one.
When in doubt, always prefer an implementation that's included in the .NET framework. You can have high expectations that such an implementation is tested by hundreds of thousands of programmers, has been carefully reviewed for security and usability and will be maintained for years to come.
The mutex approach is an easy one to get going. It however suffers from a pretty severe security problem. A denial of service attack is very simple to get going, you cannot keep the name of your mutex a secret and anybody can trivially create a mutex with the same name and prevent your program from ever starting up.
The process name approach is deeply flawed for the same reason. There is no guarantee that a process name is unique. Not just easy to exploit but easily triggered by accident.
WindowsFormsApplicationBase has an image problem in the eyes of C# programmers. They choke at the namespace name and assume that their program will somehow be infected with vb-isms. That's nonsense, it is just a plain .NET class that's useable in any language.