here is one comparison I have found helpful. Essentially BRIEF and ORB are much faster. There is not a good comparison of scale invariance there but personally I have found SURF/SIFT to be more scale invariant than BRIEF and ORB. I recommend if you are going to use these for a specific use case you try both to see which meets your needs best.