This study presents a methodology to evaluate and prevent security vulnerabilities issues for web applications. The analysis process is based on the use of techniques and tools that allow to perform security assessments of white box and black box, to carry out the security validation of a web application in an agile and precise way. The objective of the methodology is to take advantage of the synergies of semi-automatic static and dynamic security analysis tools and manual checks. Each one of the phases contemplated in the methodology is supported by security analysis tools of different degrees of coverage, so that the results generated in one phase are used as feed for the following phases in order to get an optimized global security analysis result. The methodology can be used as part of other more general methodologies that do not cover how to use static and dynamic analysis tools in the implementation and testing phases of a Secure Software Development Life Cycle (SSDLC). A practical application of the methodology to analyze the security of a real web application demonstrates its effectiveness by obtaining a better optimized vulnerability detection result against the true and false positive metrics. Dynamic analysis with manual checking is used to audit the results, 24.6 per cent of security vulnerabilities reported by the static analysis has been checked and it allows to study which vulnerabilities can be directly exploited externally. This phase is very important because it permits that each reported vulnerability can be checked by a dynamic second tool to confirm whether a vulnerability is true or false positive and it allows to study which vulnerabilities can be directly exploited externally. Dynamic analysis finds six (6) additional critical vulnerabilities. Access control analysis finds other five (5) important vulnerabilities such as Insufficient Protected Passwords or Weak Password Policy and Excessive Authentication Attacks, two vulnerabilities that permit brute force attacks.
Software security is a very important feature to be considered in the software development processes resulting from the evolution of technology, information and services [
Frequently, the lack of security is conceived by the lack of security measures taken by software engineers [
Software security in companies or organizations that develop information systems has to be constituted as an intangible asset and an important feature for which development companies are recognized, offering customers a set of standards and good practices to apply in their projects in order to guarantee and provide the necessary reliability on the data and sensitive information they handle. To establish a management system for information security, consider the three pillars of McGraw [
The popularity of web applications, as a solution to interconnect users with data and services [
There are several methods and ways of evaluating software security, among which the use of Applications Security Testing (AST) stands out, they are one of the resources most widely used by developers to guarantee the security of applications [
The software evaluation process represents an important human and technical effort, due to the degree of depth involved in carrying out each type of security analysis [
Software security vulnerabilities are one of the most critical problems and one of the greatest concerns related to computer security. The method analyzes its strengths and weaknesses and takes advantage of the synergies that both types of analysis have to optimize the result of the security evaluation of a web application.
For the development of the methodology, we investigated the characteristics of static and dynamic security analysis, the tools available to carry them out and their use in isolation. The objective is to get an appropriate knowledge that can be used to combine them in several phases, feed the results between them and correlate the partial results obtained in each phase to maximize the overall result of the security evaluation of a web application.
To evaluate the methodology and its applicability to any web application, we used the CMS Made Simple case study [
Finally, the results classified by the metrics included in the methodology will be evaluated and published, in such a way that the experimental security analysis presents a first scope of the effectiveness of the methodology.
This paper makes the followings contributions:
It presents a new, light, repeatable and flexible methodology that allows evaluating the security of any type of web application, being independent of the type of development and implementation technology. It uses two different security analysis perspectives that allow having an internal software evaluation surface using SAST tools and an external one with DAST tools. In addition, manual verification criteria are defined, which allow identifying and minimizing the false positives produced by the tools, and identifying false negatives that, due to the scope of the applications, often cannot be detected in a timely manner. Demonstrates that the proposed methodology is simple to apply and effective in evaluating the security of web applications. Furthermore, its ease of understanding allows the methodological process to be used in any traditional or agile development context or practices of integration and continuous delivery. It shows how the methodology can be used by others more general methodologies that do not explain how to combine static and dynamic security analysis tools in web applications. It can be integrated into both traditional and agile SDLC.
The rest of the paper is organized as follows. Section 2 is a background of security testing technologies, Section 3 describes the new testing methodology approach, Section 4 is a practical application of the methodology, and Section 5 reviews the related work about web applications security testing tools, conclusions and future work.
Web applications are software products that are accessed from a web browser and operate under a client-server architecture that communicates through the HTTP communication protocol (see
According to Ginige et al. [
Although web applications have facilitated many tasks, their exposure and access to all types of end users has created new threats [
Web applications are subject to continuous changes in usage trends, implementation technologies and new security vulnerabilities, as shown by the results of the 2017 survey conducted by Stack Overflow [
Given the growing trend in the use of new technologies for the development of web applications and the need to mitigate security weaknesses, research papers such as [
There is not a perfect security analysis technique, normally, it suffers from generating alarms for security vulnerabilities that do not really exist (false positives) and from not detecting some security vulnerabilities (false negatives) [
In the context of SSDLC, Static Application Security Testing (SAST) is a white box type vulnerability analysis technique, which bases its process on evaluating the source code to detect possible security vulnerabilities.
Its main advantage is the ability to find errors and coding problems in web applications that produce security vulnerabilities without having to run the application [
Manual code security analysis is arduous and complex if the number of lines of code derived from the process of software development is high, which would mean a longer evaluation time [
SAST (Static Application Security Testing) tools build a source code model to perform a static process to evaluate and identify security vulnerabilities in early stages of development, in order to generate products software with fewer weaknesses from the beginning of code implementation [
The method described in [
SAST tools still have a wide margin for improvement, both their internal design and the set of vulnerabilities they can detect determine their degree of effectiveness. Each of the tools have a different degree of coverage and, in general terms, the commercial tools prevail in the market and have a higher degree of effectiveness than free tools, as shown by the studies [
Dynamic analysis is a security vulnerability detection technique that includes black box and white box testing. This type of analysis makes a check of the running software [
A dynamic black box analysis makes an evaluation directly with the Frontend of the application using fuzzing test in the entry inputs of an application, in order to determine and identify vulnerabilities and weaknesses in the architecture of web applications [
There are basically two categories of dynamic analysis tools:
Dynamic Application Security Testing (DAST). Interactive Application Security Testing (IAST) and real-time protection (RASP), are white box tools and can complement the work done by the black box type identifying security vulnerabilities.
DAST (Dynamic Application Security Testing) tools try to discover the components, forms of a web application in a first phase. Once the web site is discovered, the tool sends many malicious requests to the input fields of the application forms and analyze the responses to see if there are security vulnerabilities [
The advantages of DAST are:
It can detect vulnerabilities in the final version of the software product. It simulates the behavior of a malicious user performing different types of attacks. It is independent of a programming language.
The disadvantages of DAST are:
It is based on trial and error techniques, so they can not cover all attack surface. It cannot detect problems related to software logic. It is complex to cover all possible attack vectors.
DAST occupy a prominent place among the activities to be carried out within the life cycle of secure software development in the penetration testing phase, see
IAST (Interactive Application Security Testing), RASP (Runtime Application Self-Protection). IAST-RASP are part of a second generation of security analysis tools. In their process of evolution some research papers [
IAST are dynamic white box analysis tools, unlike SAST, which are static tools. The advantage of IAST, as explained by Rohr [
According to the studies of Sureda et al. [
Hybrid analysis is a type of security analysis that emerges from the combination of static, black and white box dynamic analysis, so we seek to take advantage of the synergies that may exist between the types of tools and extend the attack surface of the application to obtain a global increase in the effectiveness of the security analysis of a web application. An example of this is the combination of tests with SAST and DAST tools, which integrate their capabilities to achieve greater accuracy in the detection of security vulnerabilities and false positives. At present, it is possible to find some combinations of tools that perform static, dynamic or interactive analysis, such as the integration of DAST-SAST [
Currently, there are security methodologies that already establish ways to measure the security of web applications based on a series of phases and security analysis steps that allow a complete perspective of security in web applications.
Shakeel [ Penetration Testing Execution Standard (PTES) OWASP Testing Guide NIST 800-115 2008 Open Source Testing Methodology Manual (OSSTMM)
PTES is a standard designed to offer security services based on its penetration testing analysis [ Interactions prior to commitment Information gathering Threat modeling Vulnerability analysis Exploitation Post-exploitation Reports
Its advantage, according to Shanley et al. [
The OWASP Test Guide is a referential framework that includes a set of control points, based on the collection of good security practices used in web applications [ Before starting the development During the design and definition During the development During the implementation Maintenance and operations
The second part contains a set of techniques that serve to identify the different security problems that a web application may suffer. The techniques specified in version four of the guide includes:
Information collection (OTG-INFO) Configuration and deployment management tests (OTG-CONFIG) Identity management tests (OTG-IDENT) Authentication tests (OTG-AUTHN) Authorization tests (OTG-AUTHZ) Session management tests (OTG-SESS) Input validation tests (OTG-INPVAL) Error handling tests (OTG-ERR) Weak cryptography tests (OTG-CRYPST) Business logic tests (OTG-BUSLOGIC) Customer side tests (OTG-CLIENT)
This methodology establishes the parts of a web application that must be tested successively using different types of tools such as SAST, DAST, etc. However, it does not delve into how to use the tools together and take advantage of the synergies that may exist between them.
This methodology is a standard developed by the National Institute of Standards and Technology of the United States [ Planning Execution Post-Execution
OSSTMM is a general security testing methodology mainly focused on audit areas [ Security of the information Process safety Security of Internet technologies Security of communications Wireless security Physical security
Each of the mentioned aspects contains a list of controls and techniques to meet the needs of the organization. It covers all security areas including web applications.
The main objective of the proposed methodology is to carry out the security analysis activities of the SSDLC through a procedure that attempts to automate the method using semi-automatic tools identified in Section 2.3 of both types: Static analysis and dynamic analysis. This methodology may be part of other more general methodologies that contemplate the use of static and dynamic analysis tools, such as, the OWASP Testing guide v4 methodology.
The combination of this type of tools SAST and DAST, was born with the intention of taking advantage of the characteristics of each one to achieve a greater degree of coverage with respect to the security analysis. Studies conducted by Ball [
The proposed approach contemplates a three-phase process at the macro level of definition, execution and results, where the core of the methodology is in the execution phase, which, in itself, its operation is fed by the results of the static analysis and dynamic. They are verified and validated through a correlation of results that allows to manually establish scenarios that, from an attacker’s perspective, allow us to discard false positives and identify possible false negatives. Finally, we emphasize on evaluating in a particular way the access control of the applications to increase the degree of vulnerability coverage that allows obtaining the greatest possible effectiveness during the analysis. The proposed methodology contemplates three phases shown in
Phase 1: Definition
Evaluation selection objective
Identification of evaluation objective characteristics
Tools selection
Metrics selection
Phase 2: Execution
Semi-automatic static analysis
Checking static analysis results with dynamic analysis
Semi-automatic dynamic analysis
Access control analysis
Phase 3: Classification of results
Analysis of results using selected metrics
The phases are detailed in the following sections.
This phase is the beginning of the security assessment, in which we define all the basic aspects of the evaluation process. The start is important because of the definition of the aspects exposed in the process (see
The process involved in this phase implies:
The establishment of evaluation objectives: It defines the web application to be evaluated, the objectives, the scope of the analysis, the evaluation criteria and the vulnerabilities that will be covered by the analysis. This step is important, as the tool selection process is highly dependent on this. Identification of characteristics of the objective: All the technical details of the web application (programming language, execution environment and implementation) are identified and the context and operation of the web application are validated. Selection of analysis tools: Based on the previous point 1.2, the SAST and DAST tools to be used are identified, and it is verified that the selected tools are compatible with the evaluation environment. Metrics definition: The metrics that will be used to classify the results are established. For the proposed methodology the metrics will be:
TP: True positive: A detection by any type of vulnerability analysis that is a real security vulnerability. FP: False positive: A detection by any type of vulnerability analysis that is not a real security vulnerability. TP percentage of total vulnerabilities. FP Percentage of total vulnerabilities.
Once the definition phase is complete, the evaluation team has the technological and functional context of the web application, the tools and the metrics that will be necessary to continue with the execution phase of the security evaluation.
The execution phase is where the security assessment process of the web applications is accomplished, it follows the process shown in
The security analysis execution procedure uses the information obtained in the definition phase to execute the corresponding analyzes, for which the following process is followed.
Semi-automatic static analysis (white-box): In this step, the security analysis of the web application to be evaluated is executed, for which it is necessary to execute the static analysis through the SAST tool selected in Step 1.3. Subsequently, the security auditor must manually audit and classify the results that the tools generate and classify them as true positives and false positives.
Checking results with dynamic and analysis: To carry out this activity, the results classified in Step 2.1 are required as input. Thus, the dynamic analysis will be carried out through the DAST tool selected in Step 1.3, with manual verification. This will allow having the first classification of security analysis results.
Semi-automatic dynamic analysis (black-box): This step is independent of Steps 2.1 and 2.2, and can be executed in parallel with the previous steps. This step will serve to identify the vulnerabilities that the static analysis does not cover. From this step, new results will be obtained that will serve to extend the degree of coverage of the analyzes carried out.
Dynamic access control analysis: This security analysis will be used to carry out security checks regarding the authentication and authorization processes of the web application.
Results: This step is fed by Steps 2.1, 2.2, 2.3, and 2.4. In this step, the degree of coverage of the security analysis executed will be analyzed, correlated and identified based on the vulnerabilities and metrics identified. It is important to mention that this step is key, since it allows to locate vulnerabilities and that the auditor, or the development team can later improve the quality [
In the security assessment process, the evaluator may face some doubts due to the scope of each one of the types of analysis executed, that is why some considerations have been established to be considered during this phase:
The established process can be iterated as many times as the evaluator believes necessary to ensure the security of the web application. In Step 1.3 of tool selection, the evaluator can select the evaluation tool that best suits the context of functional and technological execution of the web application, and can also choose how to use the tool based on the best performance of the tool. The identification of the TP and FP metrics. See the related job section for tool comparisons. See related work section for tool comparisons. The execution process is flexible; that is, it allows us to use or integrate another or different types of analysis to the satisfaction of the evaluator. When selecting tools, one should investigate the degree of coverage of the vulnerabilities for which they are designed and the differences between the types of analysis and the tools to cover the greatest possible attack surface. A vulnerability database checking process such as NVD (National Vulnerability Database) must be performed to discover CVEs (Common Vulnerabilities Exposures) in relation to third-party software. In addition, a static code security analysis process must be performed to determine its degree of exposure. The results phase must contemplate minimally the detailed structure in its corresponding phase.
In this final phase, we apply the selected metrics to the results of execution phase to classify the security vulnerabilities detected. Complementary statistical graphs and detailed information of the identified vulnerabilities will be shown to clarify the results of the security analysis based on this methodology. In execution phase, the information is organized according to the following data:
Static analysis of source code (white-box)
Audit Results Results classification based on defined metrics Confirm of the static analysis results with dynamic analysis
Confirmed security vulnerabilities Not confirmed security vulnerabilities Results classification based on defined metrics Dynamic analysis (black box) and Dynamic access control analysis
Audit Results Results classification based on defined metrics
Results of security vulnerabilities classified by metrics
This section summarizes the methodology application process developed in order to measure the degree of accuracy in the identification of security vulnerabilities. The application process is detailed in the following sections according to each of the phases proposed in the methodology.
In this phase, according to the proposed methodology, we define all the basic aspects of the evaluation process:
Scope of evaluation: Evaluate security vulnerabilities resulting from tools that are classified as high/critical. Evaluation objective: CMS Made Simple v2.2.1 Characteristics of the application Type: CMS or Website content manager. Programming language: PHP Application Server: Apache Database Management Server: MySQL Analysis tools selected SAST: SCA Fortify. DAST: OWASP ZAP The metrics to use in this methodology are:
Vulnerability enumeration and classification Common Weakness Enumeration: CWE. TP: True positive: A detection by any type of vulnerability analysis that is a real security vulnerability. FP: False positive: A detection by any type of vulnerability analysis that is not a real security vulnerability. TP percentage. Percentage.
In this phase, for the experimental process of applying the methodology, a test environment was prepared using the service manager for XAMPP that contains a package of services and applications that is used to run PHP web applications, then, the selected application was installed in a local environment as part of objective evaluation.
Furthermore, for this process, it is important to explain that the tools were selected based on the information mapped for the case study, considering the scope of the study and the technological context of the web application to be evaluated. It is important to emphasize that the methodology is capable of adapting to any technological context, therefore, based on the evaluation objective identified in Phase 1, the tools that best suit the auditor’s needs can be selected.
Initially, through the SAST tool, the evaluation of the source code of the evaluation objective was configured. The analysis in this step generated the results shown in
In summary, the results included 403 vulnerability issues of critical type, 84 of high type, 34 of medium type and 214 of low type were obtained, of which, according to the scope of the study, critical vulnerability issues were evaluated, the same as detailed in
CWE | Detections |
---|---|
CWE-79 XSS persistent | 44 |
CWE-79 XSS reflected | 244 |
CWE-94 dangerous file inclusion | 20 |
CWE-676 dangerous function | 2 |
CWE-95 code injection | 5 |
CWE-91 JSON injection | 2 |
CWE-321 hardcoded encryption key | 1 |
CWE-601 open redirect | 16 |
CWE-259 hardcoded password | 2 |
CWE-22 path manipulation | 48 |
CWE-359 privacy violation | 17 |
CWE-89 SQL injection | 1 |
CWE-497 system information leak | 1 |
After obtaining the results, the audit was carried out for each of the vulnerabilities, obtaining the first classification of security vulnerabilities.
In this step, each of the identified and classified vulnerabilities of the results generated by the SAST tool was checked in two steps:
For each security vulnerability identify audited by SAST is performed a dynamic analysis penetration testing technique using the selected DAST tool. Once the active scan has finished a manual verification of each vulnerability is performed to classify it as a true positive or a false positive.
According to the aspects mentioned in relation to the execution phase, there were incidents of security vulnerabilities identified by SAST that did not enter within the range of action or attack surface of DAST due to the different depth and characteristics of each type of analysis, so, in those cases, the result of SAST was considered. The security vulnerability audited in
The analysis performed using the DAST tool, which is required in a first instance, human support performs a manual crawling of the web application to identify the entire attack surface of the web application. The tool selected for analysis (OWASP ZAP) was configured as an interception proxy, in order to visualize and record all manual crawling traffic.
Once the attack surface was identified, an automatic active scan was performed using the DAST tool. This scan injects recursively distinct payloads or malicious strings in order to exploit and discover vulnerabilities in the input fields of the web application. This step is very time-consuming. The tool in the active analysis process detected five types of high-level vulnerabilities, six medium-type and 13 low-level.
Finally, the vulnerabilities result of the active scan were audited manually with the help of DAST tool on the web site. The results of the manual audit of High security vulnerabilities are shown in
CWE | file | DAST results |
---|---|---|
CWE-352 anti CSRF token scan | login.php | TP1 |
CWE-79 XSS | addgroup.php | TP |
CWE-79 XSS | adduser.php | TP |
CWE-79 XSS | editgroup.php | TP |
CWE-79 XSS | edituser.php | TP |
CWE-79 XSS | myaccount.php | TP |
CWE-79 XSS | moduleinterface.php | TP |
CWE-79 XSS | siteprefs.php | TP |
CWE-89 SQLi | ajax-content.php | TP |
CWE-89 SQLi | moduleinterface.php | TP |
CWE-89 SQLi | edituser.php | TP |
CWE-89 SQLi | moduleinterface.php | TP |
CWE-540 source code disclosure | adduser.php | FP |
CWE-540 source code disclosure | listbookmarks.php | FP |
CWE-540 source code disclosure | listgroups.php | FP |
In this phase, two access control checks are carried out:
Dynamic analysis using the selected tool of type DAST. The Owasp ZAP tool has an access control plugin to check accesses to all web application’s URLs. Brute force attack using a list of payloads extracted from the OWASP SecLists Project [
CWE | Results |
---|---|
CWE-20 improper input validation | TP1 |
CWE-522 insufficiently protected credentials | TP |
CWE-307 improper restriction of excessive authentication attempts | TP |
CWE-521 weak password policy | TP |
CWE-564 SQLi hibernate XSS | TP |
Access control security analysis conclusions:
CWE-20. Brute force attack detected that there is no validation regarding discrimination between uppercase and lowercase letters in the user field. CWE-522. The credentials are transported in plain text and are not encrypted in any of the layers of the web application. CWE-307. Access control does not have a security policy of blocking requests that limit brute force attacks. CWE-521. There is no password creation policy, any password can be configured. CWE-564. The web application does not validate or sanitize the entries, resulting in the access control breaking.
Next, the results obtained mainly in the execution phase of the methodology classified by each of the steps proposed are detailed below.
The percentage of true positives obtained is 89.33, compared to a 10.66 percent of false positives (see
CWE | Detections | TP1 | FP2 |
---|---|---|---|
CWE-79 XSS-P | 44 | 44 | 0 |
CWE-79 XSS-R | 244 | 238 | 6 |
CWE-94 | 20 | 9 | 11 |
CWE-676 | 2 | 2 | 0 |
CWE-95 | 5 | 5 | 0 |
CWE-91 | 2 | 2 | 0 |
CWE-321 | 1 | 0 | 1 |
CWE-601 | 16 | 16 | 0 |
CWE-259 | 2 | 2 | 2 |
CWE-22 | 48 | 25 | 23 |
CWE-359 | 17 | 17 | 0 |
CWE-89 | 1 | 1 | 0 |
CWE-497 | 1 | 1 | 0 |
Total | 403 | 360 | 43 |
The vulnerability with the highest number of detections in this step was persistent and reflected type Cross Site Scripting (CWE-79), these vulnerabilities are the ones that most affect the analyzed web application.
CWE | SAST detections | SAST TP | SAST FP | DAST TP | DAST FP | N/A by DAST |
---|---|---|---|---|---|---|
CWE-79 XSS persistent | 44 | 44 | 0 | – | – | 44 |
CWE-79 XSS reflected | 244 | 238 | 6 | 90 | 6 | 142 |
CWE-94 dangerous file inclusion | 20 | 9 | 11 | 9 | 0 | 2 |
CWE-676 dangerous function | 2 | 2 | 0 | – | – | 2 |
CWE-95 code injection | 5 | 5 | 0 | – | – | 5 |
CWE-91 JSON injection | 2 | 2 | 0 | – | – | 2 |
CWE-321 hardcoded encryption key | 1 | 0 | 1 | – | – | 0 |
CWE-601 open redirect | 16 | 16 | 0 | – | – | 16 |
CWE-259 hardcoded password | 2 | 2 | 2 | – | – | 2 |
CWE-22 path manipulation | 48 | 25 | 23 | – | – | 25 |
CWE-359 privacy violation | 17 | 17 | 0 | – | – | 17 |
CWE-89 SQL injection | 1 | 1 | 0 | – | – | 1 |
CWE-497 system information leak | 1 | 1 | 0 | – | – | 1 |
Total | 403 | 360 | 43 | 99 | 6 | 298 |
Based on the considerations of the methodology, in this analysis there were vulnerabilities detections that could not be validated through dynamic analysis, so
In
The results of the DAST audit (see
CWE | File | DAST manual audit |
---|---|---|
CWE-79 Refleted XSS | addgroup.php | TP1 |
CWE-79 Refleted XSS | adduser.php | TP |
CWE-79 Refleted XSS | editgroup.php | TP |
CWE-79 Refleted XSS | edituser.php | TP |
CWE-79 Refleted XSS | myaccount.php | TP |
CWE-79 Refleted XSS | siteprefs.php | TP |
CWE | File | DAST manual audit |
---|---|---|
CWE-352 anti CSRF | login.php | TP1 |
CWE-79 R-XSS | moduleinterface.php | TP |
CWE-89 SQLI | ajax-content.php | TP |
CWE-89 SQLI | moduleinterface.php | TP |
CWE-89 SQLI | edituser.php | TP |
CWE-89 SQLI | moduleinterface.php | TP |
CWE-540 | adduser.php | FP2 |
CWE-540 | listbookmarks.php | FP |
CWE-540 | listgroups.php | FP |
One (1) CSRF, four (4) SQLI and one (1) Reflected XSS. The results confirm that independents SAST analysis find more vulnerabilities that isolate the DAST analysis: The SAST audited analysis confirm (360) vulnerabilities and the DAST audited analysis confirms (12) vulnerabilities (6 common with SAST and 6 non-common with SAST).
CWE | TP1 | FP2 |
---|---|---|
CWE-20 improper input validation | 1 | 0 |
CWE-522 insufficiently protected credentials | 1 | 0 |
CWE-307 excessive authentication attempts | 1 | 0 |
CWE-521 weak password policy | 1 | 0 |
CWE-564 SQLi hibernate | 1 | 0 |
All discovered access control vulnerabilities confirm that it is possible to gain access to the application and subsequently gain additional privileged access to other resources of the application such as files or database objects. The access control analysis has found few vulnerabilities, but they are all really critical.
Based in the results obtained, we formulate four research question to validate the methodology practical application:
The methodology takes advantage of using the main skills of SAST and DAST tools to combine them with the aim of reducing the number of false positives and finding a greater number of true positives. This objective is reached by checking SAST results using a DAST tool that includes manual checks and correlating the results of the isolation DAST analysis with the previous results obtained by the SAST analysis checked by a DAST tool that includes manual checks. Besides the methodology performs access control checks using several plugins of DAST tool and manual checks to discover additional vulnerabilities.
The revision of the state of the art demonstrates that there are no other methods to use different types of tools on the market to take advantage of the main skills of the SAST and DAST tools in combination. The state of the art reveals that there are specific hybrid SAST-DAST tools, but there is no general and repeatable methodology that allows a process such as the one presented in this study to be followed to analyze the security of a web application that takes advantage of the synergies of the analyzes performed. In this study, in addition, this methodology presents the necessary mechanism how to adapt to any process of software development, technology or web language. It is also not closed, which allows one or more analysis activities to be included to improve the effectiveness obtained. Our methodology is general and open to incorporate any SAST tool to be adapted to any web language application server and also it is open to incorporate any DAST tool for combining with the SAST tool. Related work shows diverse performance comparatives of tools [
We have selected a real web application, CMS Simple used by many users on the Internet, to demonstrate the degree of performance of this new methodology. The results of the practical application of the methodology In
Analysis | Detections | TP | FP | %TP | %FP |
---|---|---|---|---|---|
SAST | 403 | 360 | 43 | 89.33 | 10.66 |
DAST-NC(DAST) | 9(15) | 6(12) | 3 | 80 | 20 |
A. CONTROL | 5 | 5 | 0 | 100 | 0 |
Based on the results obtained from the implementation of the evaluation methodology, the degree of coverage of the security analysis methodology for web applications is presented below. Considering the number of true positives and false positives, it was found that the degree of effectiveness is 88,9 percent based on the identification of vulnerabilities classified as true positives and false positives (see
The static analysis (white box) finds many security vulnerabilities with few false positives (10,69 per cent) identified and confirmed by the posterior auditory of each reported vulnerability.
Dynamic analysis with manual checking is used to audit the results, 24,6 per cent of security vulnerabilities reported by the static analysis has been checked and it allows to study which vulnerabilities can be directly exploited externally. This phase is very important because it permits that each reported vulnerability can be checked by a second tool (DAST) to confirm whether a vulnerability is true or false positive.
Dynamic analysis (black box) finds six (6) additional critical vulnerabilities as SQL injections (4), CSRF (1) in login.php page, XSS (1).
Access control analysis finds other five (5) important vulnerabilities, such as Insufficient Protected Passwords or Weak Password Policy and Excessive Authentication Attacks, two vulnerabilities that permit brute force attacks. The access control vulnerabilities found are very critical because they allow brute force attacks and XSS or SQLI to steal session state or application credentials.
XSS is the most frequent vulnerability, 288 different XSS vulnerabilities are found and classified as critical by SAST and DAST tools. However, in general, all vulnerabilities found as a result of the practical application of the methodology are included in the OWASP Top Ten 2017 project as the most critical vulnerabilities.
The application of the methodology confirms that their true positive number (371) is better than the true positive number obtained by only SAST analysis (360) due to the methodology combines the DAST analysis (6) and access control analysis (5). Also, the methodology performs auditory steps to reduce the number of false positives as much as possible in the phases:
4.2.1 (SAST analysis) It confirms 43 FP 4.2.2 (Audit SAST results with DAST and manual checks) It confirms 90 SAST TP/9 SAST FP 4.2.3 (DAST analysis) It confirms 6 FP 4.2.4 (Access Control analysis) It confirms that all vulnerabilities discovered (5) are TP
This method can be extended to any other web application you consider to select a SAST tool capable of analyze the source code language used by the web application and a DAST tool selected according to diverse performance comparatives as showed in [
General methodologies to analyze the security of web applications as the OWASP testing guide version 4 explains the types of tools to use to analyze each part of a web application, client, server side, database server, application code, configurations, input validation, authentication, sessions, authorization, logging, etc., but it does not explain how to take advantage of the use of different types of tools in combination. Our methodology can be integrated as a part of OWASP testing guide to analyze each part of a web application that can be analyzed with SAST, DAST and manual checks according to the OWASP testing guide. Thus, these two methodologies can benefit each other to achieve a better result in their joint.
This methodology in comparison to other proposals how [
The flexibility of the presented methodology shows that it can be applied to any method of Software development. In SDLC, it can be shown that the methodology can support the processes and activities carried out in each of the phases to ensure the security of the software being built. For example, in SDLC it should be used in the development, implementation and testing phases so that its results validate the context of the security of the web application before going to a production stage. In the same way with WATERFALL, the process of evaluation and correlation of results allows that the results obtained in each phase of software development can be used as an security criterion in each change of stage so that the levels of security are guaranteed during the process. However, despite its easy adaptation to any context, we emphasize, in the processes of agile development and continuous deliveries. The simplicity that our methodology demonstrates makes it a validation and assurance mechanism for web applications in an agile development environment, since the incremental process and the small launches that are made at the end of each Sprint in SCRUM, allow them to be analyzed and validate at a more specific level of detail than with other software development methods, in addition, the acceptance criteria of each User Story can be used to determine the levels of security through the definition of criteria or stories of the attacker how It is mentioned [
In this section we include related works about techniques to combine static and dynamic analysis security tools and studies that analyze and compare Static Application Security Testing (SAST) and Dynamic Application Security Testing (DAST). All the methodologies analyzed first perform a SAST analysis to generate a set of test cases to test each vulnerability found by SAST with a DAST tool for TP verification. These methods try to automatize the task of eliminating the false positives detecting by previous SAST analysis.
Our methodology also uses DAST tools to confirm SAST detections, but our method performs manual audits of independent SAST and DAST analysis and finally correlates all analysis results to present a complete report. We demonstrate that manual audit of vulnerabilities with the help of tools is also necessary to verify all vulnerabilities detected by SAST and DAST tools. Moreover, our repeatable method permits select any SAST or DAST and IAST tool to adapt to any web technology and vulnerability categories.
Kim et al. [
Csallner and Smaragdakis describe several implementations of this type of tools. Despite its simplicity they can find bugs that would require complex static analysis efforts. Check ‘n’ Crash [
Babic et al. [
PHP vulnerability hunter [
Dogan et al. [
Prokhorenko et al. [
Kiss et al. [
Nunes et al. [
Díaz [
Lakshmi et al. [
Babincev et al. [
Skoruppa [
Le et al. [
These studies will permit to select the most suitable SAST, DAST or IAST tools to best adapt to any web technology and vulnerability categories. Wagner et al. [
According to Elizabeth Fong [
The study of Ware et al. [
The work of Diaz et al. [
The OWASP Benchmark project [
CWE | Detections | TP 1 | FP 2 |
---|---|---|---|
FindSecBugs v1.4.6 | 96.84 | 57.74 | 39.10 |
FindBugs v3.0.1 | 5.12 | 5.19 | -0.07 |
OWASP ZAP v-2016 | 19.95 | 0.12 | 19.84 |
PMD v5.2.3 | 0.00 | 0.00 | 0.00 |
SonarQube Java Plugin v3.14 | 50.36 | 17.02 | 33.34 |
Commercial SAST-01 | 28.96 | 12.22 | 26.74 |
Commercial SAST-02 | 56.13 | 25.53 | 30.60 |
Commercial SAST-03 | 46.33 | 21.34 | 24.89 |
Commercial SAST-04 | 61.45 | 28.81 | 32.64 |
Commercial SAST-05 | 47.74 | 29.03 | 18.71 |
Commercial SAST-06 | 85.02 | 52.09 | 32.93 |
It includes security test cases for XSS, SQLI, XPATH, command injection, path traversal and other sensitive data exposure vulnerabilities included in OWASP top ten 2017. It has computed results for six commercial tools (SAST, DAST and IAST), but is not publicly available due to licensing reasons. Analyzing the project, it is difficult to obtain conclusions about SAST commercial tools because their results are anonymous. The project provides the average results of all commercial tools including tools categories, SAST tools (HP-Fortify, Checkmarx, Coverity, IBM-Appscan, Parasoft Jtest), DAST (Arachni, ZAP, Acunetix, Burp, HP-Webinspect, Rapid7) and IAST (Contrast). The results of 5 SAST free tools, PMD, FindBugs, FindSecurityBugs, SonarQube and OWASP ZAP (dynamic black box tool) are available against version 1.2 beta of the Benchmark. SonarQube and FindSecurityBugs obtain the best results with a similar balance between true and false positives. FindSecurityBugs finds more vulnerabilities but has higher rate of false positives. Actually, SonarQube besides its Java plugin integrates PMD, FindBugs and FindSecurityBugs plugins for increasing the detection ratio of security vulnerabilities. Java and PMD plugins of SonarQube are focused in quality bugs not in security vulnerabilities, therefore SonarQube relies in FindBugs and FindSecurityBugs to find security vulnerabilities.
Bau et al. [
In this study, the development and execution of an evaluation methodology based on the combination of static, dynamic analysis and access control techniques was carried out, which left the following conclusions. This methodology is repeatable and flexible allowing it to be adapted to any language, or web technology by selecting the SAST tools adapted to specific languages and selecting the DAST tools adapted to specific web technologies, such as input vectors, authentication methods, etc.
The application of this methodology to analyze the security of a web application shows that the static analysis finds many security vulnerabilities with few false positives (10,69 per cent) identified and confirmed by the posterior auditory of each reported vulnerability. Dynamic analysis with manual checking is used to audit the results, 24,6 per cent of security vulnerabilities reported by the static analysis has been checked and it allows to study which vulnerabilities can be directly exploited externally. This phase is very important because it permits that each reported vulnerability can be checked by a second tool (DAST) to confirm whether a vulnerability is true or false positive. Dynamic analysis finds six (6) additional critical vulnerabilities as SQL injections (4), CSRF (1) in login.php page, XSS (1). Access control analysis finds other five (5) important vulnerabilities, such as Insufficient Protected Passwords or Weak Password Policy and Excessive Authentication Attacks, two vulnerabilities that permit brute force attacks. The access control vulnerabilities found are very critical because they allow brute force attacks and XSS or SQLI to steal session state or application credentials. The results obtained from the security tests carried out revealed that the analyzed web application presents cross Site Scripting (XSS) vulnerabilities in most cases, due to a weak process of verification of data entries by the application.
The joint use of the SAST and DAST tools to carry out each step of the proposed methodology allowed the security testing process to be more productive and effective. The dynamic analysis tool is used first to manually verify the results of static analysis to effectively discard false positives, following a dynamic analysis is performed and finally, the results of the static and dynamic analysis are correlated obtaining an improvement in the effectiveness of the results in terms of true and false positives.
The methodology used in this study only evaluates the application as such, it is important that security implementations are involved in all layers of the same or that the deployment of a web application integrates all possible levels of security.
This methodology can be integrated as a part of OWASP testing guide to analyze each part of a web application that can be analyzed with SAST, DAST and manual checks according to the OWASP testing guide. Thus, these two methodologies can benefit each other to achieve a better result in their joint.
In the security analysis process of web applications, there are a lot of security techniques, however, new ways of violating applications are still being discovered today, so it would be important to establish a formal method to teach developers that involves tests, standards, norms, good practices and not only language teaching and ways to program, in order to minimize the security risk presented by the human factor. In addition, emphasis should be placed on the use of the secure software development life cycle for all software projects since the SSDLC already involves security testing in each of the development phases and discard methods that propose the execution of security tests in the final phases of software development. The flexibility of the presented methodology shows that it can be applied to any method of Software development. In SDLC, it can be shown that the methodology can support the processes and activities carried out in each of the phases to ensure the security of the software being built. For example, in SDLC it should be used in the development, implementation and testing phases so that its results validate the context of the security of the web application before going to a production stage.
The developed methodology generated a high and precise degree of effectiveness, which makes it an alternative to be used in any software development approach, especially in agile processes and continuous delivery, where deliveries are expected to be partial and in a short time. In this way, the proposed methodology would allow members of the development team to carry out the necessary security validations in each of the planned launches. Furthermore, it can be integrated into other methodologies that suggest using the same types of analysis, but do not specify how to do it. This methodology details how to integrate diverse types of analysis to optimize results. The practical application shows that the integrated analysis SAST, DAST and manual obtain better results.
The methodology, like any other, can be improved, its improvements include the implementation of analysis tools such as IAST, which according to some research presents much more accurate results when detecting vulnerabilities, among its results is the minimization of false positives or the scope of coverage of improved vulnerabilities among other essential features that support the security verification process. IAST tools could be integrated into the semi-automatic dynamic analysis subphase of the execution phase (see Section 4.2.3). The combined work of the SAST, DAST, IAST tool types aided by manual checks required to audit the tools’ individual results will result in the highest detection of vulnerabilities with the lowest number of false positives. In addition, we will study the possibility of including in the method the combination of two SAST tools, two DAST tools and two different IAST tools in each case. Combining two tools of the same type improves the results due to their different designs in terms of the algorithms they use.