Why is Mobile Testing So Hard?

  • 16-Oct-2013

We often find enterprises underestimating the effort required to test mobile apps. After all, how much testing effort will it take if you are developing native apps for just two operating systems --  iOS (Apple's mobile operating system that runs iPhone and iPad) and Android?

A lot, it turns out.

Let's look at iOS first.

The latest version of iOS (version 7) runs on iPhone 4, 4S, 5 and the recently released 5C and 5S. All these phone models have differences -- e.g., iPhone 5x has a bigger screen size compared to iPhone 4's. From there differences run to capabilities such as screen resolution, camera capabilities, fingerprint recognition, and so forth. Of course, you can chose to ignore all the additional capabilities, but if you really want to use them, you'll also need to test your app epxeriences on iOS devices that don't have those capabilities.

Of course, iOS doesn't just run phones. It's also on iPads (again multiple versions), iPods, and even Apple TV. Plus there are older versions of these devices -- such as iPhone 3GS and iPad 1 that are still used in many markets globally.

This issue of device diversity becomes bigger in case of Android. In lieu of one, single build a' la iOS, you'll find multiple builds of Android for any given release. Apart from the stock build, each handset manufacturer and many telcos have their own builds, complete with proprietary bells and whistles.

And, finally, as compared to iOS, the differences in terms of screen sizes, capabilities, and limitations are even more pronounced because every manufacturer wants to try to differentiate their devices.

To be sure, creating just two version of a native app -- one for iOS and one for Android devices -- may be a good strategy, since by targeting just these two, you are covering close to 90% of your mobile users. Just make sure you don't underestimate the testing effort, and account for specialized mobile testing (including hardware testing) in your application development life cycle. Also, when evaluating Enterprise Mobile Technology, find out what sort of development tooling (including debugging, device emulators, and testing)  the tools provide and if you can integrate them with other 3rd-party testing tools.

We cover how well different vendors perform here (among other criteria) in our recently released Enterprise Mobile Technology report.

Other Enterprise Mobile Technology posts

MD