The TAB is OCC’s Technical Advisory Board, made up of representatives from OCC’s project and product groups. The TAB’s role includes reviewing technology developments and their impact on the company’s software development.
Test your production code
Things can go wrong between debug and release builds:
- conditional compilation
- linking
- incorrect project configuration/build dependencies
This means we need to test the production build. However, this is not always as simple as it sounds. Apple makes it almost impossible to test release builds due to the effort involved in arranging provisioning profiles and signing certificates.
How can we make this easier? Read on.
Beta programmes
All the app stores provide beta programmes, offering a way of (relatively) simply deploying apps to a group of users for testing.
- alpha channel for early builds
- beta channel for more stable builds
- choose from invite-only testing or open to anyone
- you can set a maximum number of testers and testers will then be able to access the app on a first-come-first-served basis
- you can roll out updates to a certain percentage of your testers and increase that percentage once you’re happy the update is stable
Apple
- Test Flight internal: available only to accounts linked to the developer’s company
- Test Flight external: invite only
- apps are available to users within the Apple Test Flight app
- test apps expire after 60 days
Microsoft
- Package Flight: invite only
- different groups can test different packages
Advantages
- the most important thing here is that you will be testing the actual build that will be deployed to the app store
- once testing is complete, it only takes a couple of button presses to publish the (now tested) package to the live app store
- the app will go through the certification process so you catch failures early
- it’s streamlined
Android
- 5 minutes to build and deploy
- available to testers ~ 1 hour
iOS
- 10 minutes to build and deploy
- available to testers ~ 10 hours (10 minutes if you choose to skip certification)
Windows
- 10 minutes to build and deploy
- available to testers ~ 5 hours
- getting a build onto multiple devices for testing takes only seconds per device
Disadvantages
- if you fail certification testing is blocked
- Windows provides no option to skip certification, but failures are rare
- with Apple, you can skip certification, failures are more common
- there is a lag to deployment so no way to test something in 10 minutes
- you need to ensure testers check the version number reported or they can test the wrong build
- test build automatically rolls out to all testers which can cause problems
- Windows allows you to control who gets which version
- Android has Beta or Alpha but little granular control within that
- Apple gives you no control at all
It works
ContrOCC Remote has been doing this for 12 months. Around 20 builds have gone through each store and the process has caught issues that could potentially have shipped.
Windows Package Flight has also allowed us to define a ‘stable’ test version that can be used for demos of the new functionality.
TAB recommendation: make use of these release testing programmes for any but the most trivial of apps; they will make the testing process quicker, easier and more robust.