Android Anti-Malware Tests Show Big Detection Differences

Anti-malware test lab AV-Test has released results for 41 Android anti-malware packages that it tested for effectiveness. The results show vast differences in performance, but there are several important caveats to the tests.

The products tested ran the gamut, from free apps made by companies you've never heard of to pricey products from big-time security vendors. The only objective of the testing was to measure detection rates. Although detection might be the most important characteristic of an anti-malware product, it isn't the only one--so the report doesn't endorse or condemn any of the products tested.

Growth of Android malware threats identified by AV-Test since January 2011.

Malware finds a welcome mat in Android because of certain decisions Google made to ease the process of developing and distributing apps. The cost and qualifications for distributing apps through Google's Android Market are lower than those for Apple's App Store. Android also lets users allow software to be installed from other stores, whereas iOS users must "jailbreak" their phones to do so. Many of the third-party Android markets are not very picky about checking the apps they accept and often offer malware, but malware has been found in Google's market, too. As a result, Google has, in recent months, ratcheted up its automated scanning of submissions to the Market. The malware that AV-Test tested for included phishing and banking trojans, spyware, bots, root exploits, SMS fraud, premium dialers, and fake installers.

[ Larry Seltzer argues that whitelisting is the best way for enterprises to keep their Android devices safe. Read about one whitelisting solution..]

AV-Test doesn't provide exact detection rates--perhaps to discourage unjustified comparisons. Instead it split the products into five detection groups: >90%, >65%, >40%, >0% and 0%. Most of the products detected more than 40% but less than 65% of the malware threats. Six found none. Seven found more than 90%. They are:

The second-tier products might well be good choices, too--the failure to detect one type of malware might not matter in certain areas.

As for the packages that found nothing, it's not clear whether they weren't functioning properly or are just bad products. None detected the Eicar test file, which is a specific non-malicious file that all products are supposed to detect and thereby prove that they are running.

AV-Test made some trade-off decisions for the testing that one has to take into account when considering the test results. For instance, in order to make it possible to run a large number of tests, AV-Test chose to use the Android emulator that comes with Google's Android SDK, set for Gingerbread (Android 2.3, API level 10). The advantage of the emulator is that it lets testing scale much more easily than on a phone. On the other hand, because the emulator is not a phone--for instance, it doesn't have a phone number--it might cause malware to fail or behave differently. Some apps would not run it at all. For those apps, AV-Test used a Samsung Galaxy Tab running Android 2.2 (Froyo) and a Samsung Galaxy Nexus running Android 4.0 (Ice Cream Sandwich).

Another possible problem is that some Android malware still falls into a gray area that is not strictly defined as malware. If an app throws up annoying ads is it malware? Some products might not think so.

AV-Test did not consider the other features a product might have, such as backup or anti-theft protection.

You can make a case that by avoiding shady stores and using common sense, you can avoid Android malware without installing anti-malware software. However, AV-Test concludes that you should at least consider running one of them. There are attacks which could get past Google, at least for a while, and some of the products work well enough that some day you might be happy you took the trouble to install them.