Android Lollipop Has Less App Crashes Than iOS 8, Says Study

Crittercism, a San Francisco based apps performance monitoring, has release a new study that will most definitely add fuel to the iOS vs. Android debate.  The report looks at Google Inc.’s (GOOG) 5.0 (“Lollipop”) and Apple, Inc.’s (AAPL) iOS 8 and asks a simple question — which is more stable?

To assess that, they compared different recent major releases of Android and iOS.  Using their developer-targeted tools which watch for app crashes, they then examined how frequently apps crash on each platform.

Surprisingly, in spite of hearing in recent months of a number of firmware and API bugs in recent months, on the software front Android Lollipop was narrowly stability king, crashing only 2.0 percent of the time, on average, over the past couple months of data.

iOS 8 (left) was found to have more app crashes than Android 5.0 Lollipop (right).
The bad news for Android fans is that two prior builds — Android 4.0 (“Ice Cream Sandwich”) and Android 4.4 (“KitKat”) — are the most unstable, with apps crashing roughly 2.6 percent of the time.  That’s bad news for Android fans as many devices are stilling shipping with KitKat, which metrics show is the most used Android distribution, at present [source].  KitKat is currently installed on just under 40 percent of Android devices — roughly 2 in 5.

iOS 8, which also had some widely publicized bugs of its own, was better than Android KitKat, but worse than Lollipop.  Its app crash rate was 2.18 percent.  One thing that makes that number more troubling for Apple is that Google’s platform appears to be getting more stable while Apple’s appears to be getting less so, from a release-on-release standpoint.

Apple’s previous build, iOS 7, saw apps crash only 1.9 percent of the time — better than iOS 8 or Android Lollipop.  Sadly for iOS fans, that remarkably stable build is fading fast in the rearview.

The good news for Apple is that over the past couple of days Lollipop crash rates actually popped up a bit as adoption increased perhaps suggesting that the slow rollout was keeping them artificially low.  Currently Lollipop’s crash rate is 2.15 and iOS 8’s rate — which has been steadily dropping, unlike Lollipop’s — is 2.18.  It remains to be seen if the two are now truly neck-and-neck or if this is just a blip on Lollipop’s part given its lower average error rate.

An important word of caution in viewing these numbers: while it may seem obvious, it’s important to point out that the study only looked at apps crashes. Smartphones can suffer a variety of other stability issues including firmware bugs, app glitches (which don’t crash the app but due something unintended), and security vulnerabilities.  Stacking up Android vs. iOS in overall stability would be a complex and somewhat arbitrary process.

Another factor worth pointing out is that some of Apple’s issues may be indirectly caused by its hardware.  While most flagship Android smartphones are shipping with 2-3 GB of LPDDR2 or LPDDR3, the iPhone 6 and 6+ shipped with only 1 GB of memory stacked inside their system-on-a-chip (SoC).  Running out of memory is one frequent source of app crashes, so that could offer one plausible explanation of why apps on the iPhone are less stable than on Android Lollipop devices.

That still leaves the question, then, of why iOS 7 was able to do so well.  It’s possible that OS process in iOS 7 consumed less memory than in iOS 8.  iOS 8 tacked on a lot of new functionality, much of which requires some degree of background processes.  Among the memory intensive features added include widgets and the Spotlight indexing service.

Indeed there’s annecdotal evidence in forums postings of this.  Some users complaining of sluggish notifications and other odd behavior while running iOS 8 on older devices like the iPhone 5S — even when no apps were running.  It looks like sometimes less is more.

On the other hand, the glass half full for iOS fans is that their distributions are more stable than the ones 98.4 percent of Android users are running.  Only 1.6 percent of Android users are currently on Lollipop, according to Google, although that number should rise quickly in the next couple months.