Which Windows Phones Have We Tested?
We managed to get our hands on four Windows Phone devices: Nokia Lumia 525, Lumia 620, Lumia 625 and Lumia 1320.
First, we ran all our tests on the last Windows Phone 8.0 distribution that was made available by Microsoft - version 8.0.10532.166. Then we upgraded all these smartphones to Windows Phone 8.1.
On Lumia 625 we upgraded to version 8.10.14147.180 while on all the other Lumias, we upgraded to 8.10.12393.890. These were the latest versions that were available when we made our measurements.
The Tests We Performed
We have used the Multibench 2 and WP Bench for general performance testing, to get an idea on how well each version used the following hardware resources: the processor, the RAM memory, the storage space and the graphics chip.
Then, we used the Sunspider benchmark to get an idea on whether users will get some performance improvements when browsing the web with Internet Explorer.
In order to measure if there are any important differences when it comes to the battery, we used the battery test from WP Bench.
Last but not least we also measured the boot time for each Windows Phone device, using a stopwatch.
Windows Phone 8.0 vs Windows Phone 8.1 - CPU Performance
We first measured the CPU performance delivered by each operating system:
The MultiBench 2 processor test has revealed very small differences between the two operating systems. In general, Windows Phone 8.1 was about 2 - 2.5% faster than its predecessor, when working with the CPU. The only exception was Lumia 1320, where Windows Phone 8.1 was 1.7% slower when using the CPU.
The WP Bench test has revealed bigger differences though. In this processor test, it is very important to get a lower score. The score was improved by Windows Phone 8.1 from 8% on the Lumia 525 up to 19% on the Lumia 620. The only smartphone where Windows Phone 8.1 did not deliver any improvements was the Lumia 1320. On this device, it was 4% slower when using the CPU.
Windows Phone 8.0 vs Windows Phone 8.1 - Storage Performance
Next, we took a look at the speed of the data transfers when using the storage space available on each device. We used the same two benchmarks:
With the exception of the Lumia 1320, MultiBench 2 has shown improvements on all devices. The biggest positive difference was noticed on the Lumia 625, where data transfers to and from the storage space were made 18% faster in Windows Phone 8.1. Luckily, on the Lumia 1320 the differences were way too small to be noticed by users.
The WP Bench revealed the same positive trend. However, the improvements it measured were smaller - up to 8% on the Lumia 525 and 9% on the Lumia 1320.
Windows Phone 8.0 vs Windows Phone 8.1 - Memory Performance
For our next comparison we looked at performance when using the RAM memory on each smartphone. This is an area where the results vary depending on the benchmark.
MultiBench 2 said that Windows Phone 8.1 doesn't deliver any performance improvements when making transfers to and from the RAM memory. Transfers were up to 4% slower in this test, in Windows Phone 8.1.
Windows Phone 8.0 vs Windows Phone 8.1 - Graphics Performance
Next we compared the graphics performance delivered by the two operating systems.
MultiBench 2 has revealed some improvements for all devices except Lumia 525 were the scores were almost identical. The biggest improvements was noticed on the Lumia 625 - aproximately 11%.
Windows Phone 8.0 vs Windows Phone 8.1 - Web Browsing Performance
Sunspider is another benchmark where it is important to get a lower score. Here too the picture was again mixed.
On some smartphones, like the Lumia 525 and Lumia 1320, Windows Phone 8.1 delivered slightly better browsing performance. The biggest improvement was of 6% on the Lumia 1320.
On the Lumia 620 and Lumia 625, Windows Phone 8.1 delivered lower performance when browsing the web. The worst results were noticed on the Lumia 625, where Windows Phone 8.1 was 22% slower than Windows Phone 8.
In terms of web browsing performance, Windows Phone 8.1 delivers mixed results. On some devices you will notice some small improvements while on others web browsing was slower.
Windows Phone 8.0 vs Windows Phone 8.1 - Boot Timings
When measuring the time it takes to start each Windows Phone, the 8.1 version delivered good improvements across the board. The biggest improvement was noticed on the Lumia 1320 where the booting procedure took 5 seconds less (15%).
Windows Phone 8.0 vs Windows Phone 8.1 - Battery Comparison
We used the WP Bench battery test which stresses each device to the maximum and measures how long the battery last. Unfortunately this test crashed on our Lumia 1320 device. This problem occurred both when using Windows Phone 8.0 and Windows Phone 8.1. There seems to be an incompatibility between this device and the WP Bench app.
On the devices that we did test, the picture was again mixed. On the Lumia 525, Windows Phone 8.1 lasted 23 minutes (11%) less. On the Lumia 620, Windows Phone 8.1 lasted 11 minutes more (6%) while on the Lumia 625 it lasted 9 minutes (4%) more.
To summarize, when compared to Windows Phone 8.0, Windows Phone 8.1 delivers clear performance improvements in the following areas: data transfers to and from the storage space and boot timings. When it comes to graphics performance, things are mostly unchanged after making the upgrade.
In areas like CPU performance, RAM memory performance, web browsing performance and battery time, you may notice improvements on some devices while on others you may notice performance degradation.
Before you go, don't hesitate to share with us your experience with Windows Phone 8.1. Did you get any performance benefits on your smartphone after upgrading? Let us know using the comments form below.