Hockey vs. Testflight

pabloDragon picture pabloDragon · Nov 8, 2014 · Viewed 13.9k times · Source

We're trying to decide between these two services for an upcoming beta. The new TestFlight looks much improved but we are still concerned about 3 things:

  1. The user experience for testers (iTunes reviews of the TestFlight app imply this can be confusing)
  2. The possibility of rejection or delay by apple when reviewing our beta releases.
  3. Limited to IOS8

Hockey, on the other hand, seems to have a better tester UX. It supports multiple OS versions. And of course, no review is required.

The downside with Hockey seems to be a limit of 100 devices: http://support.hockeyapp.net/kb/client-integration-ios-mac-os-x/adding-new-devices-to-your-provisioning-profile

Here's our comparison grid: https://docs.google.com/spreadsheets/d/1CuYlsLsZPW-79hEre7jLppfwQpG4WmW3fDvHvIJ86wY/edit#gid=0

Would appreciate any feedback.

Answer

ocwang picture ocwang · Nov 8, 2014

We've been beta testing our App via both HockeyApp and Apple's new Testflight within the last few weeks as well. I would recommend using both in parallel and seeing the pros and cons for yourself and from there you can choose one over the other. Here are our insights from the last few weeks:

HockeyApp Pros:

  • no need for an approval process
  • quick upload of new versions while maintaining access to old versions

HockeyApp Cons:

  • requires a bit more work to initially set up each beta tester (need to have their device registered with HockeyApp so that you can register their UDID with a provisioning profile and then a new archived build needs to be uploaded to HockeyApp containing that updated provisioning profile)
  • Only 100 tester slots (although unless you're really close to app store submission, you probably don't even need 100 spots)

TestFlight Pros:

  • Don't have to deal with UDID or provisioning profiles, strictly emails only
  • 1000 10000 slots
  • Now includes sharing of public links, so that other testers outside your organisation can use to download your beta builds.
  • Beta Approval process is pretty short (1.5 days for us)

TestFlight Cons:

  • Need iOS 8 to install TestFlight
  • Only 25 internal tester slots
  • Only can have one active build at a time Now supports having multiple active builds