The SWAMP team prides itself on having a dedicated cybersecurity group. We take this responsibility very seriously. As proud as we are, it would be foolish to not seek review by someone unaffiliated with our project that can provide an objective assessment. So when the reputable cybersecurity firm Black Hills Information Security (BHIS) generously offered to perform a network penetration test, web application penetration test, and risk assessment all pro bono, we jumped at the opportunity. BHIS is owned by John Strand, one of the co-hosts of the popular Paul’s Security Weekly podcast.

The pen test planning started with our staff providing a high level overview of the SWAMP network and DNS namespace to determine what resources would be considered in-scope and to plan the order in which the resources would be tested. It also gave us an opportunity to announce maintenance windows during times when user facing services would be tested. SWAMP users were notified of these windows in advance of the testing. However, the SWAMP’s infrastructure was designed to handle significant network loads and was not disrupted by the pen test activities. The actual pen testing started on January 9th, 2017 with a reconnaissance phase in which BHIS attempted to discover as much information about SWAMP staff, resources, and names as possible through social media, websites, DNS registration records, and other public records. From the earliest days of the SWAMP, our staff has given attention to these types of public information vulnerabilities, and BHIS recognized that with praise in their final report.

After the reconnaissance phase, BHIS began scanning the SWAMP’s external network for listening services and checking for known vulnerabilities in any services found. Fortunately, the network scan discovered no surprise exposures that we were not already aware of, although it is quite common to find vulnerable services running during such a scan. The SWAMP staff performs these types of scans on a regular basis.

During the following week of the test, BHIS focused on the SWAMP web application and backend services to check for unique vulnerabilities in the web application. This was the most valuable part of the test, because although we run software analysis tools on the SWAMP code to try to detect weaknesses, the current state of software analysis is somewhat limited, which is something SWAMP is working to change. There can be logic mistakes that are difficult to detect using automated tools and require manual investigation. Their manual scan found such a weakness in the form of a privilege escalation vulnerability that was easier to detect through pen testing than it was using static analysis tools. Instead of waiting until delivering the final report, BHIS immediately and confidentially contacted SWAMP’s security team to report the problem. We fixed it in test and production the same day and released a new version of SWAMP-in-Box the following week. That BHIS decided to notify us immediately, demonstrated their understanding of the risk presented by the vulnerability. It also gave them an impression of the effectiveness of our incident response procedures, which they rated as excellent.

They also performed an internal network scan, which included using a pivot host that represented a compromised system on our internal network. This simulated a more threatening scenario for an attack. The scan found a few hosts that were behind on patches, which were due to delayed updates from license re-negotiations. The tests also provided us a way to effectively see how well our Intrusion Detection System (IDS) worked. During the test, our IDS detected and notified us of a variety of port scans, SQL Injection attacks, SSH brute force attempts, network traceroutes, unusual email traffic, and malware downloads. On each new class of alerts, we confirmed with the testers that the alarms were due to their activity. After comparing the detection and notification logs with the information in the BHIS report, we determined that the IDS detected and notified us for the majority of their attempts.

The test concluded on January 27, 2017. The final report was that the overall risk level of the SWAMP is “Low”, which is the second lowest rating on their five category scale.

From the final report: “BHIS considers the overall risk to SWAMP based on the key findings in this report to be Low. SWAMP’s external web application appeared to have secure coding standards in place and the external testing did not result in any significant issues. BHIS found the internal network to have several systems missing up-to-date patches, specifically in virtual machine management devices.”

The SWAMP team is already working to address the issues that BHIS brought up and plans to resolve them within the next month, if not sooner.

The SWAMP team would like to thank Black Hills Information Security for this penetration test. The fruits of their efforts not only protect the SWAMP, but also its users, and thus contributes to achieving a more secure software community.