Performance-Based Comparative Assessment of Open Source Web Vulnerability Scanners

Journal Article
Alsaleh, Mansour . 2017
Magazine \ Newspaper: 
Security and Communication Networks
t to 16
Publication Abstract: 

The proliferation of malicious content on the web and the rapidly growing demand for defending web-based systems motivated quality assurance practitioners to use cost-effective web penetration testing tools. Although the performance of leading commercial web vulnerability scanners has been benchmarked and validated, the effectiveness of open source dynamic testing tools has gained little attention in the literature. The high cost of commercial web vulnerability detection tools and the lack of sound evaluation of open source scanners contributed to a sharp decline in both static and dynamic testing of web content. This paper experimentally evaluates the features and the performance of a set of popular open source web vulnerability detection tools. By analyzing the crawling capabilities of the chosen scanners and identifying their performance shortcomings, we aim at providing a baseline for aiding designers of open source scanners in building effective tools. Several trusted security benchmarks were utilized to perform a quantitative comparative performance assessment of 4 open source web vulnerability scanners. As a case study, 140 unique web-based applications were also scanned using some selected scanners from those included in the comparative evaluation. While the observed vulnerability detection accuracy of the majority of the evaluated scanners is high, our findings identify some fundamental limitations in the crawling capabilities of these scanners. The scanners included in the conducted case study agreed that 12.86% of the scanned web-based systems were vulnerable to Structured Query Language (SQL) injection attacks whereas Cross-Site Scripting (XSS) vulnerabilities were discovered in 11.43%. After demonstrating the ways in which different performance properties of one scanner might correlate with each other and highlighting the inconsistencies between the results reported by different scanners, we emphasize on the importance of addressing the problem from a software engineering perspective and we provide recommendations for helping developers to improve their tools’ capabilities.

PDF icon 6158107_1.pdf1.34 MB