Vulnerability scanning vs. Penetration testing: comparing the two security offerings

This blog was written by a third party author.

It’s no secret: the number of security vulnerabilities organizations must contend with is overwhelming. According to a 2019 Risk Based Security report, there were 22,316 newly-discovered vulnerabilities last year. One Patch Tuesday disclosed a record number of 327 vulnerabilities in a single day. Just keeping up is becoming a monumental task.

But knowing where and how your organization may be vulnerable is critical to maintaining a healthy security posture. As vulnerabilities add up and the threat landscape widens, two crucial strategies for understanding where you are and where you need to be security-wise are vulnerability assessments and penetration tests.

At the very core, almost all organizations should be doing both. If you’re not, you may be exposing yourself to great risks.

It’s easy to understand why some may confuse the two strategies (they are complimentary, after all), but there are key differences between vulnerability assessments and penetration testing.

The differences between vulnerability scanning and penetration testing

Vulnerability scanning is typically conducted with software leveraging automated processes and looks for known vulnerabilities in various systems. Once complete, a report on risk exposure is generated. Penetration testing (or pen tests), on the other hand, leverages manual processes and is typically carried out by a cybersecurity expert or experts that try to find holes and exploits within your system architecture. Penetration testing is sometimes referred to as ethical hacking, in that you are enlisting the help of a third party to “hack” into your systems to see if they are easily penetrable.

Vulnerability testing determines the extent to which critical systems and sensitive information are vulnerable to compromise or attack due to outstanding patches and / or common security misconfigurations. Penetration testing takes this a step further to exploit the vulnerabilities identified in order to gain access to critical systems, sensitive information, or a specified trophy. While automated vulnerability scanning can help you identify security flaws that need remediation, it can’t holistically help you evaluate the strength of your organization’s security controls against complex strategies a human attacker might employ. For instance, chaining multiple vulnerabilities together to leverage them as a part of the overall kill chain.

Here’s an analogy that underscores the difference between the two strategies. If your systems were a car and the threat landscape were rough roads and icy conditions, a vulnerability scan would represent the vehicle’s 10-point check — tires, suspension, engine, etc. A pen test would represent the equivalent of taking the car on a test drive down a rough road in bad weather to see how everything holds up.

It's important to remember that a pen test isn't just capitalizing on vulnerabilities that a vulnerability scanner would discover. Pen tests dig deeper into those configurations and interactions between devices and systems (and where they are located) that can be exploited.

There are many cases in which your environment “passes” a vulnerability scan without any identified issues but could still be insecure. You wouldn’t know this without a proper pen test.

Why perform vulnerability scans or pen tests?

New vulnerabilities are discovered and disclosed every day. While compliance mandates or basic security strategies may dictate that you need to patch at least monthly, vulnerability scans executed more frequently are recommended. This way, organizations can benefit significantly by gaining an accurate representation of their security profile.

Depending on the complexity of the vulnerabilities, some exploits can start appearing in the wild rather quickly. Zero-day exploits (when vulnerabilities are exploited in the wild immediately) happen more often than we’d like. If you’re not performing vulnerability scans consistently and following up with remediation, you're exposing yourself to potential threats. 

Once you establish a scan cadence and remediate where possible, you will have a good baseline of your security and compliance posture.

Based on the results from vulnerability scans, it’s a great idea after a couple of the scan cycles to introduce pen testing. The advantages of pen tests for an organization are based on the fact that vulnerability scanners are limited to identifying specific vulnerabilities that are present on any particular asset. The true risk of those vulnerabilities may or may not be fully realized until a pen tester tries them within a specific environment.

Instead of looking at just one vulnerability, pen testers will leverage multiple or chain multiple vulnerabilities together for a bigger effect. Something that may not be rated as all that severe during a vulnerability scan may become a linchpin to a more insidious exploit when chained with multiple vulnerabilities in specific environments.

The current threat landscape presents so many risks that organizations can’t afford to miss out on the power of leveraging these two strategies that should go hand in hand.

Are there any risks when doing a pen test?

First, there’s no significant risk involved in performing vulnerability assessments. But remember: the scan is only as good as the threat intelligence and vulnerability data you put into it. The most perceptible risk is in not doing it frequently enough.

But when it comes to pen tests, there is a serious risk of which to be aware. The most common risk stems from who a company chooses as its pen tester. If the pen tester is not experienced or lacks the understanding of the toolsets, compliance requirements, and real-world exploits of what's going on in the wild, your results will not be accurate.

An inexperienced or flawed pen tester can represent both compliance and security risks to an organization.

Having or hiring a capable and qualified pen tester is critical, as they will help you contextualize the results based on your environment and understand and prioritize your remediation.

Finally, we must understand that both the pen test and vulnerability scans are a “moment in time” assessment. It bears repeating that they both must be done regularly.

Can organizations do their own pen tests?

In almost all cases, penetration testing should be performed by an external provider or third party. Depending on the organization's size and maturity, some companies are in a strong position to carry out their own penetration tests with an experienced internal team.

But those companies are few and far between. Even for businesses that do possess the internal capabilities will often have their pen tests augmented and validated against a baseline with a third party. Plus, a third party may be more likely to uncover vulnerabilities that a company might not even think to test.

For many organizations and industries, compliance requirements and mandates require pen testing to be carried out by a third party anyway.

Ultimately, a robust vulnerability scan or penetration test is only as good as the threat intelligence or vulnerability insights you put into it. When looking for a third party to help you with these critical security strategies, make sure they bring lots to the table: robust methodologies and toolsets, a vast library of known vulnerabilities and security misconfigurations, and demonstrating a full understanding of what's going on in the wild and how things are being exploited.

The best pen testers should be able to evaluate your attack surface, satisfy compliance requirements, understand real-world exploitation of your staff, understand and prioritize remediation, and work with security experts.