Security for everyone - How we review security products
Even though there are plenty of available antivirus solutions, home computer security is far from being perfect. One of the main reasons is that, in the feature rush, many developers forget about the most important piece in the puzzle: the user. They design complicated security solutions with lots of features which normal users don't understand and have no clue about how to use them. Our team has been on a mission to find those security solutions which are effective in protecting your computer from malware and easy to use for most, if not all people. In this article we’d like to tell you about our approach to the “Security for everyone” series, describing how we test each product and explain what each of our badges means:
What is Digital Citizen doing?
Traditionally, reviewers have been focusing on the features and the capabilities of security software. We want to do it differently.
We know that a correctly configured program will be able to cope with all the security requirements of an average user. You probably don't need intelligent packet filters, tunneling features, logging features and fancy activity monitors. You probably just want to block suspicious programs and make sure that your computer is safe from malware and hacker attacks. This is what we are focusing on: How practical is a program in securing your computer? Not what its potential is, but how useful to someone who does not want to put much effort into using it or learning anything about it. We want to find the perfect security program for your grandmother or any other family member who thinks that browser tabs are hard to cope with.
There is a logical reason behind this: most users are unlikely to need most of the advanced or "extra" features security programs offer. Furthermore, most users are unlikely to invest too much of their time in actively securing their systems. And even when they do invest enough time, they probably do not like it. As a consequence, it is only fair to assume that most users require accessible software with robust basic features that are easy to configure.
What kinds of security products are we testing?
We are testing mostly commercial products that are distributed in the form of security suites. These products generally offer complete protection and we can evaluate the efficiency of all the important protection modules: firewall, antivirus and antispyware. Running only an antivirus or just a firewall is no longer enough in today’s cyberspace. You need to have at least these three protection layers running on your computer, coupled with anti-rootkit protection and real-time HTTP traffic scanning.
How we do our security tests
During the years, our reviews have evolved a lot in terms of completeness, length and quality. Today we are structuring our reviews in five important test areas:
- The download and installation experience - We first evaluate the supposedly simple exercise of downloading and installing the security product that we are reviewing. We look at the number of steps required to download and install the product, the information required so that you are allowed to use the product, the size of the download and number of reboots required to finalize the installation. We are also interested in whether the security product updates itself shortly after the installation and how friendly the first interactions are.
- Integration with Windows, web browsers and universal Windows platform apps - Many software developers tend to ignore the new features that are introduced in Windows. To make sure users have a good experience, we start by checking if a security suite automatically disables the Windows Firewall and Windows Defender . Having two firewalls running on your system can cause headaches for users. And Windows Defender is likely to cause conflicts if it runs simultaneously with other antivirus products. Then, we look at how much a security suite affects the startup time of our test computer, how much memory is needed by the security suite’s processes in order to run and how well it integrates with networking features such as the Homegroup. Last but not least, we test whether you can kill the running processes of the security suite and how well it is integrated with popular web browsers like Google Chrome, Mozilla Firefox, Microsoft Edge and Opera.
- Ease of use & configuration - We start our more advanced tests in this section. We first look at the options available for scanning your system for malware, the default scheduled scans and the available scheduling options. We evaluate how easy it is to find your way through all the menus and configuration options, without needing to access the Help documentation. Then, we check how easy it is to get access to the support services offered by each product, its Help documentation and reporting features, if they are available. We also try to evaluate how easy it is to understand the alerts and messages shared by the security product we are reviewing. Will people need a manual to understand what the security product is saying, or is it using plain English to communicate with users?
- Firewall protection - We use two network vulnerability scanners in order to evaluate the performance of the firewall: Nmap and Nessus. We look for vulnerabilities to certain types of network attacks, open ports and their relative importance to the security of your system, and any other information that can be easily identified through a routine scan: MAC address, operating system, etc. We also check if the security suite is able to assign the correct security profile when connecting to a new network, so that you get the best balance between security and access to network sharing features (when connecting to safe, private networks).
- Antivirus and antispyware protection - We start by visiting a couple of malicious websites to see if the security suite is protecting the user while browsing the web. Then, we check how each security product handles external memory drives. We place several malware samples on a USB memory stick, we plug it into our test computer and then we evaluate what the security suite does. For our most complex anti-malware test, we make a clean Windows 10 installation on our test computer, we implant malware on it and then we ask the security product that we are reviewing to install itself and remove all the malware. We try to evaluate how good the antivirus engine is, depending on how efficient it is at removing the malware, but also how fast and how easy the entire procedure is. We also run a series of full system antivirus scans and monitor how fast the antivirus engine is at scanning your computer. We also look for a second opinion and we check what other independent security organisations have had to say about the efficiency of the products that we are rating. It’s good to get a second opinion, isn’t it?
- Extra features - Many IT security companies develop lots of extra features for their security products. We install and use each and every extra tool that comes with a certain product. While doing so, we analyze how useful these extra tools are and whether they bring added value to the customer. Because we consider it an important security feature for many parents, we look whether a Parental Controls module is available and whether it is easy to setup and use. We also, look for Safe Browsing features when doing online transactions or for the inclusion of a VPN service.
- Default settings you might want to adjust - Some security suites have the power to protect you efficiently but they do not, because they lack the proper default settings to do that. When we test a security suite, we also look for those settings that could be adjusted in order to make you safer. For instance, many security suites can schedule automatic antivirus scans but they don’t. If we find such settings, we list them in our review and we recommend the appropriate actions.
We try to be as complete as possible and provide a good overview of the efficiency of each product. While we don’t have a mathematical formula we use to give the final verdicts, we look at how well each product performs in all of these areas and share our recommendations.
Verdicts and what they mean
We use a five star rating system in order to share our product recommendations:
- 5 stars - These are the best of the best, top of the top, la creme de la creme. You get the picture! They are security products that we would recommend to anyone, anytime. The tools included by these security suites provide you with an excellent level of protection and they do it in a manner that is great , both easy to understand and to use. Security products can’t get better than this.
- 4 stars - These are very good security products. We recommend these security suites to everyone, because they are top notch products that are very good at protecting you against all kinds of cyber threats. Four star products are those products that are “almost great” but they stumble because they have a few small flaws.
- 3 stars - These are good security products. We consider them average solutions and we can recommend them only to people who know what they’re doing. Some of them can be very good in one a rea, like having a very good antivirus engine, while being weak at others, like being unable to provide decent firewall protection. Some products may have weak default settings which can be improved by the user. When you see a 3 star rating, it is important that you read our verdict and the pros and cons we highlighted, so that you better understand the strengths and weaknesses of the product that we are reviewing.
- 2 stars - These are security products that are mediocre at be st . They are also security products that we don’t like and we don’t recommend to our readers. These products fail to protect their users against threats and, usually, they are also hard to use and understand.
- 1 star - These are products that you should never use. They are very bad at what they should be doing: protect their users from malware and network attacks.
Besides our five stars rating system, we sometimes also give badges to those security products that shine in one or more areas :
- Editor’s Pick - When we give this badge, we want to express the fact that we really like the product that we have reviewed. This badge is given only to products that are way above average and that are also easy to use. In general, we use this badge only for security products that have received a rating of 4 or 5 stars.
- Best Security - This badge means that the security product we have reviewed offers a great level of protection to its users. These products shine in terms of effectiveness but they might not be as great in other aspects, like usability or performance. This badge can be given to products with a rating of 4 or 5 stars. Sometimes, a product with a rating of 3 might receive this badge, but only when its flaws are not related to the effectiveness of its anti-malware protection.
We hope that this series will remain useful to our readers. We don’t want you to purchase useless security tools and waste your money, while putting your security and privacy in danger. With this series, we are doing our best to identify those security products that are worth considering even on your clueless friends’ and relatives’ computers and devices.