Penetration testing, or, at a minimum, vulnerability scanning, is now a well-regarded industry best practice to improve information security. But what is it?

Overall, the best definition of penetration testing is hacking with permission. A penetration tester employs the same techniques available to hackers to see whether your network is vulnerable to them. It’s a true dual-use technology.

The phases of a penetration test usually include:

• Determining the test’s scope and rules
• Intelligence gathering
• Vulnerability analysis
• Exploitation
• Post Exploitation
• Reporting

Determining the test’s scope and rules and reporting are not part of the strict definition of penetration testing because they are not things a hacker would do, but they are crucial. Some of the choices you must make in defining a specific test are things like:

• Will it be external or internal?
• Will it search the Internet, and possibly other sources, to discover where to attack your network, or will it rely on a set of IP addresses you supply?
• How far will the test go in exploiting the vulnerabilities that are uncovered?
• How much help will the penetration tester provide for remediating the discovered vulnerabilities?

Varieties of Penetration Testing

The two main types of penetration tests are external and internal. An external penetration test begins on the internet and attempts to find and exploit vulnerabilities in your network or applications. External penetration testing is often called unauthorized testing because the tester has no more credentials than a hacker would have. Internal penetration testing is often made simpler by providing the tester with administrative credentials.

The main argument in favor of external testing is that the tester is in the same position as a real hacker. Any success the penetration tester has in identifying and exploiting vulnerabilities is the same one could expect a skilled hacker to have, which is a pretty good argument for getting the vulnerabilities fixed. The main argument in favor of internal testing is that protection shouldn’t stop at the infrastructure’s perimeter but also include vulnerabilities of systems within the perimeter. Ideally, you’d protect devices inside the network so that even if one machine were compromised, other devices would not be.

Secondary arguments for only doing external testing are:

• It limits the risk of damaging activities on the network. In the hands of even an experienced tester, the testing procedures can sometimes cause software to crash.
• It requires less trust of the penetration tester, because you are not providing key credentials.
• It probably will cost less, because it will involve fewer systems and therefore less work.
• If you have a very simple infrastructure, because of the size of your organization or because all the important stuff is in the cloud, you may be less concerned about the internal safeguards, compared to the periphery, so it may not be worth the additional cost.

In its most basic form, external penetration testing is only vulnerability scanning. What a vulnerability scan does is to examine all the possible points of access to your network and to identify vulnerabilities associated with those points. These could be an open port, a login protected by a generic password, out-of-date software that can be exploited, etc. The vulnerability tester then generates a report based on the scanner’s findings, including suggestions for remediating each vulnerability.

Since vulnerability scanning uses automated tools, many of which have excellent reputations, it is relatively inexpensive to do. You may even be able to learn how to do it yourself. Two good vulnerability scanners are Tenable Nessus and Rapid7 Nexpose. A free, open source vulnerability scanner is OpenVAS, although some do not consider it as good as the commercial products.

Vulnerability scanning is usually a good enough starting point for compliance with vulnerability management requirements, so why do the other parts of penetration testing at all? You may not have to, but here are some of the advantages that fuller penetration testing provides.

Intelligence Gathering

When you do a vulnerability scan, you specify the IP addresses you want the scan to cover. A penetration test, however, will begin by searching the internet to uncover those IP addresses and more. It may find additional IP addresses you did not know were facing the internet. It may also find other information, such as staff names and email addresses, likely passwords, telephone numbers, hostnames, names of vendors or contractors, or other useful stuff.

Intelligence gathering is also helpful by hackers using social engineering to get in. Good spear phishing, for example, benefits from email addresses for receivers and senders, employee names and nicknames, identities of vendors, etc. It can even turn up favorite hobbies of staff and send an email related to that hobby. It’s nice, therefore, to know what your organization or employees have made easily available.

Exploitation and Post-Exploitation

Exploitation refers to running the malicious code that takes advantage of the vulnerability. Usually what this will give you is access to the machine on which the vulnerability is running. Once you have access to one computer or other device, you can see what else is on the network and perhaps even have automatic access to other machines, depending on the privileges of that machine’s user. In other words, it gives you a starting point for taking over the entire network, but only a starting point.

Vulnerability scanners sometimes identify vulnerabilities that don’t matter. It may, for example, come across an operating system, accessible through a port, that has been reported to have a certain vulnerability, but the way you are using that system doesn’t allow the vulnerability to be exploited. You can determine whether the threat is real by examining it after it has been reported, or you can reduce the number of false positives you must check out by having the penetration tester attempt to exploit the vulnerability and test it for you.

In addition to reducing the false positives, exploitation can also demonstrate (by supplying screenshots or credentials, for example) how serious the vulnerability is. Sometimes this kind of evidence can be important in convincing management of the importance of vulnerability management.

Post-exploitation refers to moving beyond the starting point to the rest of the network and undertaking activities such as exfiltration, also called pillaging, which involves removing important data from your servers. Handing samples of such data to an executive can be dramatic, but it may not be necessary.

Post-exploitation also shows how other protections in your systems, such as identity management, can be bypassed. For example, acquiring the right administrative credentials can allow you to bypass protections in directory services such as Active Directory.

Reporting

For you, the most valuable part of the penetration test will be the report you get of its findings. This should include an inventory of all the vulnerabilities found, relevant information about them, including screenshots of relevant access, such as to login pages that can be exploited with password cracking programs. It should identify the vulnerability by its standard name and propose a way to remediate it, such as identifying the availability of a patch. The report should also include a good accounting of exactly what was done to get the results.

Frequency of Testing

Another aspect of the how-much-testing question is how often. Ten years ago, an annual vulnerability scan was considered adequate. Then it was every six months. Now you hear quarterly, monthly, weekly and even constantly. Probably the best answer is that it depends on things like the size of your infrastructure, how often it changes, and how well you keep up with your patches. In most cases, CGNET’s approach is that for an average small network, a semi-annual test is best, provided that you have other security protection, such as scanning for advanced persistent threats, operating as well.

Getting rid of vulnerabilities is important. There are lots of automated threats out there that can probe for them and exploit them. It’s not the whole story, however, because social engineering can get malware onto your network through a user, because unwise use at a public network can compromise credentials, or for many other reasons. It’s necessary, but not sufficient. If you’re interested in CGNET’s approach to advanced persistent threat scanning, you can find it here. We also offer phishing testing and end-user training.