The Best Benchmark Apps for Windows Phone Smartphones
We do want to test more and more Windows Phone devices here at 7 Tutorials. In order to evaluate their performance in a measurable and comparable way, we asked ourselves: which are the best apps for benchmarking Windows Phone devices? We did a bit of research and lots of testing and compiled this list with the best benchmark apps. Read on to learn which benchmarks are worth installing on your smartphone.
AnTuTu Benchmark - Free
Despite its strange name, AnTuTu is a well known benchmark due to its presence on Android. Its developers have tried to introduce the same kind of tests as on Android, while taking into consideration the limitations and quirks of the Windows Phone platform. This benchmark is available for Windows Phone 7.5, 7.8 and 8.
AnTuTu tests your phone's processor (CPU), its graphics chip (GPU), its RAM memory, the database input output (Database IO - not sure what exactly is tested here and how) and the storage memory. You can easily turn on and off the tests you want to perform.
I have run the full series of tests. I found that the testing time is quick (on my HTC 8X test phone it always took less than a minute). At the end, the results are shown as a series of points for each type of test and a total score. Unfortunately these points have little meaning for the user and there is no explanation of what they mean. You won't know what constitutes a good or a bad score. You will also notice that there is a small typo: CPU interger instead of CPU integer.
There is no way for users to compare results with others, at least not from the benchmark itself. Therefore you have no idea how well your phone performs compared to others.
We ran AnTuTu and all others, on each of three different phones, to notice how the results obtained vary between test runs. We noticed that the results obtained vary a lot from test to test. For example, on the same phone (HTC 8X), the difference in values between the best and the worst CPU integer test results was 13%. This is enough bias to make it hard to compare the performance of different phone models. On older or lower end phone models, the variability of tests results descended to 10% - an improvement but still a big variability between test runs on the same phone.
Even though the benchmark runs smoothly and is able to test all Windows Phones out there, we found it difficult to interpret test results and compare our phone's performance with others.
Benchmark Free - Free
Benchmark Free is available for all Windows Phone versions and, as its name implies, it is free. It tests your smartphone's processor, its RAM and storage. When you execute the benchmark, it runs each test 21 times and displays the average score. This is great, as it eliminates any weird variability that could appear and, at least in theory, it should display replicable test results for your phone.
The scores obtained don't really give you any idea of how good or bad your phone is. The test results are very abstract.
Fortunately, the benchmark allows you to view the test results obtained by other Windows Phone devices, so that you can compare your phone's performance with them.
Unfortunately this benchmark is not useful at all, at least not for Windows Phone 8 devices. It seems to have been designed for Windows Phone 7.5 and it was not updated since. All three phones I ran it on, obtained the same score - 7.9, even though their hardware configuration is a lot different.
Benchmark Free needs to be updated, so that it can be used to compare modern Windows Phone 8 devices. In its current form it provides value only for testing older Windows Phone 7.5 devices.
MultiBench 2 - Free
MultiBench 2 is a great free benchmark for Windows Phone. It works on all versions of this operating system and you can test literally any Windows Phone device with it.
As any good benchmark, it allows you to customize which test to run. I chose to run all tests.
The testing time is longer than when using other benchmarks, but it should be over in 3 to 5 minutes, depending on your phone. The results are shown as individual scores, for each component that was benchmarked: the processor, storage, RAM and graphics chip. The results are abstract so you won't know what they mean and if they are good or bad.
However, MultiBench 2 has a few features that will help with this. First, it stores the history of your testing. Therefore you can see the results you obtained in older test runs and see if the performance of your phone has deteriorated.
Then, you can also compare your phone's test results with the MultiBench 2 online database.
So far, MultiBench 2 has many Windows Phone 7.5 devices in its database. If you have a Windows Phone 8 device, you will see it outperforming everything. In time this should change and the charts should also include Windows Phone 8 devices.
In terms of variability of test results, we found it great. The scoring differences between test runs on the same phone were less than 1%. This means the measurement algorithm is solid and you can use MultiBench 2 to accurately compare the performance of different phone models.
PhoneMark Pro - Commercial ($1.49)
PhoneMark Pro costs $1.49 and it runs on all Windows Phone versions. Unfortunately it runs only in landscape mode and it features some annoying ads, even though it is a commercial product. Talk about trying to monetize an app.
PhoneMark Pro benchmarks your device's processor, RAM, graphics chip and storage. For each, it shows an abstract score and and overall PhoneScore which is an average of the individual scores.
A nice touch is that you can compare your phone's results with other phones and get some cool information like the operating system version popularity among the users of this app or some interesting trends information.
The tests are executed very quickly which means they are shallow in their evaluation. You will notice that the more you test your phone, the better the results obtained. During some test runs it displayed results which were 100% better than previous runs. This only means that the evaluation algorithm has not been designed well and it cannot be used to reliably measure the difference in performance between different phone models.
PhoneMark Pro is a weak benchmark app that needs to be further developed in order to be useful.
WP Bench - Commercial ($0.99)
WP Bench runs great on all Windows Phone versions and costs only $0.99. It is the most complete benchmarking app I have found on Windows Phone. It tests your phone's speed, its display and battery. In fact, it is the only benchmark that has a battery drain test. It pushes your phone to its limits and measures how long it takes for the battery to die when it the phone is stressed to its maximum. A very useful test indeed, which revealed some surprising facts about some of the phones we tested.
WP Bench is also the only benchmark which tries to help you evaluate the quality of the color reproduction of your display. It displays some patterns and shares what you should expect as the ideal results.
It is not measurable in hard numbers but it gives a good idea on the quality of your phone's display.
The speed test evaluates the phone's processor, RAM and storage and its graphics chip. It takes a short time to run and, at the end it displays first an overall score. You can publish that overall score and compare it to other users. Then, you get access to individual scores, for each hardware component.
WP Bench shows some very meaningful scores for each independent test. For, example it tells you that your phone's RAM is able to transfer data at 39.39MB/s or that the average frame rate obtained by your gpu is 33 frames/second. This is useful information, meaningful to the user even without comparing it to the test results of other users.
As any other good benchmark, WP Bench allows you to see the results obtained by other users as well some interesting statistics with the most popular devices tested with WP Bench, the most popular manufacturers of Windows Phones that were tested and the most popular operating system versions.
In terms of variability, the most well thought out test is the gpu one. It always generates the same result, no matter how much you test the phone. Other tests have some variability - up to 3% in our tests. It is small enough though to make the test results meaningful and usable in comparing the performance of different phones.
The Best Free Benchmark - MultiBench 2
Out of all the free benchmark apps for Windows Phone, MultiBench 2 has proven to be the best. Not only it is reliable in its measurements (the most reliable benchmark I tested on Windows Phone) but it also allows for easy comparison with other devices and remembers the history of your testing, so that you have an idea on how your phone's performances evolves in time.
The Best Commercial Benchmark - WP Bench
WP Bench is the best benchmark app for Windows Phone. Its price is affordable to anyone and its tests are very relevant to any phone user. Also, it is the only benchmark which doesn't just share abstract scores as test results. It shares meaningful data and measurements which give you a good idea on how your phone performs. Highly Recommended!